CONTINUOUS SYMMETRIZATION VIA POLARIZATION 
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Abstract. We discuss a one-parameter family of transformations which changes 
sets and functions continuously into their (fc, n)-Steiner symmetrizations. Our con- 
struction consists of two stages. First, we employ a continuous symmetrization 
introduced by the author in 1990 to transform sets and functions into their one- 
dimensional Steiner symmetrization. Some of our proofs in this stage rely on a 
■ simple rearrangement called polarization. 

OA ' In the second stage, we use an approximation theorem due to Blaschke and Sar- 

, vas to give an inductive definition of the continuous (fc, n)-Steiner symmetrization 

^ ' for any 2 < k < n. This transformation provides us with the desired continuous 

pL^ ' path, along which all basic characteristics of sets and functions vary monotonically. 

The latter leads to continuous versions of several convolution type inequalities and 
Dirichlet's type inequalities as well as to continuous versions of comparison theorems 
for solutions of some elliptic and parabolic partial differential equations. 

< 

^ ■ 1. Introduction 



The first geometric transformation bearing tlie name symmetrization was introduced 
by Jacob Steiner in 1836 [30] in one of his attempts to find a rigorous proof for the 
classical isoperimetric problem. Let C be a closed contour on enclosing a domain D 
. and let mD{x) denote the Lebesgue measure of the intersection of D with the vertical 

CD \ line Vx = G : — oo < y < oo}. Then Steiner's symmetrization of D with 

respect to the x-axis is defined by 



> 



o 



: (1-1) D* = {{x, y) e : \y\ < (l/2)mz)(x)}. 

' This implies, in particular, that D* is symmetric with respect to the x-axis and convex 

in the y-direction. Let C* = dD* be the boundary of D* . Steiner used his symmetriza- 
tion to show that 

■ (a) area!) = area -D*, (b) length C* < length C, 

which implies the classical isoperimetric inequality 



area D ^1 



(length C) 2 - 47r' 
assuming the existence of a minimizer. 

This ingenious idea of Steiner has appeared to be extremely fruitful and was exploited 
over the years by many authors, who proved numerous, so-called, isoperimetric in- 
equalities for several important geometrical and physical quantities characterizing the 
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shape of planar and solid regions. We want to mention the following four such in- 
equalities, for the transfinite diameter d{D) that is equal to the logarithmic capacity 
cap (D), for the inner radius r(D, a) of D at its point a E D, for the torsional rigidity 
P(D), and for the principal frequency A(D): 

(c) d(lF) < d(D), (d) r{D*,a*) > r{D, a) for every aeD, 

(e) P{D*) > P{D), (f) X{D*) < X{D). 

The first period of history of symmctrization was summarized in the classical mono- 
graph of George Polya and Gabor Szcgo "Isopcrimctric inequalities in mathematical 
physics" [24]. This fundamental study of isoperimetric inequalities was filled with new 
ideas, results, and problems, some of them still remain open. The story of continuous 
symmctrization also has its source in this book. The following question was raised in 
Note B of [24]: 

Is it possible to define a transformation Tx on D depending in a continuous way on 
the parameter A, < A < 1, such that the following conditions are satisfied: 

I. To is the identity. 

II. Ti is the transformation replacing C by the symmetrized curve C* , that is, T\ 

is Steiner's symmctrization with respect to the line I. 
111. For every A, < A < 1, has the same effect as described under (a)-(d). 

Although Polya and Szcgo mentioned only relations (a)— (d), a similar question about 
inequalities (e) and (f) falls into the same context. 

The authors of [24] did not explain explicitly what motivated them to study this 
problem. Among obvious reasons we want to mention the following three: 

• First, from the point of view of classical mechanics it is interesting to embed D 
and D* into a continuous path, along which all basic geometrical and physical 
characteristics of shape vary continuously and monotonically. 

• Often better estimates, than those provided by inequalities (b)-(f), are needed. 
Indeed, even for simple shapes, for example, for the rhombus having angle 
a = 7r/100 centered at a = 0, the gap in each of the inequalities (b)-(f) 
exceeds 40 %. 

• Steiner's symmctrization changes a given shape globally into a symmetric one. 
So, this transformation will not work in problems, where the minimizer does not 
possesses a global symmetry and in problems concerned with local minimality. 



Polya and Szego themselves studied this problem. In collaboration with M. Schiffman 
[24, Note B], they presented one such continuous transformation and proved the rela- 
tions (a)-(d) for the case of convex domains. Suppose that Z? is a convex domain on 
bounded from below and above by the graphs of functions y = yi{x) and y = y2{x), 
respectively, such that yi{x) < y2{x) for sll a < x <b. For a given continuous function 
.^:[a,;3]^[0,l],let 

(1-2) yi = yi-(t>{t) — ^ — , y2 = y2-nt) — - — 

and 

(1.3) D'''^ = {{x,y) eM.^: a<x<b, y{<y<yi}. 
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If (f) is an increasing homeomorphism from [a,/?] onto [0, 1], then it is clear that for- 
mulas (1.2), (1.3) define a continuous path from D into D* . Choosing (f)(t) = t, Polya 
and Szego [24] proved that the relations (a)"(d) hold true for all convex domains. 
S. Abramovich [1] used a variant of Polya-Szego's continuous symmetrization to prove 
monotonicity of eigenvalues of certain second order differential equations in one vari- 
able. 

Another continuous transformation, again for smooth convex domains, was introduced 
by A. McNabb in 1967 [22]. His transformation, called the partial Steiner symmetriza- 
tion, can be defined as follows. We quote from [22]: 

" A partial Steiner symmetrization of D may be performed in the following way. If 

the constant a lies between certain limits {ai < a < Q!_r), the line x = a will intersect 
the curve defined by midpoints of the line segments composing D. If just those line 
segments which have their midpoints to the left of x = a are translated parallel to 
themselves until these central points lie on x = a, the ends of the segments now 
define a curve Ca bounding a partially symmetrized region Da- It is as though the 
line X = t swept across the x-y-plane from t = —oo to t = a and the midpoints of 
the line segments became attached to the line as it passed over them. As t increases 
from a = ul to a = aji, D continuously evolves through a sequence Dt of partially 
symmetrized regions to its Steiner symmetrization D* . " 

As the author noted in [22], his goal was to demonstrate on simple examples how his 
transformation works. So, the treatment in [22] was heuristic and technical "difficulties 
were glossed over" there. 

We also want to mention two interesting continuous transformations discovered in [31] 
and [23], but those are not related, at least not directly, to the problem raised in [24]. 

The first continuous transformation into Steiner symmetrization, which works for non- 
convex domains and satisfies all the requirements of the Polya-Szego problem, was 
introduced by this author [27] . Our continuous symmetrization, which we will abbre- 
viate as SC symmetrization, can be considered as an extension of McNabb's partial 
Steiner symmetrization for the case of non-convex domains.^ We want to emphasize 
here that the approaches used in [22] and [27] are different. 

It is interesting to mention that, eventually, the original idea of Polya and Szego was 
developed by F. Brock [12], [13], who defined a continuous symmetrization, called 
BC symmetrization in this paper, which works for non-convex domains. This was 
achieved by choosing a parametrization 0(t) = 1 — e^*, — oo < t < oo, in (1.2), 
(1.3), combined with some other innovations. Instead of abbreviations SC and BC, 
we sometimes write "Solynin's continuous symmetrization" and "Brock's continuous 
symmetrization", respectively. One particular difference between SC symmetrization 
and BC symmetrization is that under the first transformation the change of the shape 
is localized near some boundary arcs while the second transformation changes the 
boundary globally. 

Although the exposition in [27] was given for planar domains, in the final Remark 5 
[27], the author emphasized that all definitions and proofs can be extended without 
substantial changes to n-dimensional spaces and that all major results of the paper 
haA'(^ n-dimensional counterparts. 

^The paper [27] does not refer to McNabb's work [22] since at that time the author was not aware 
of McNabb's pubUcation. 
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The primary goal of the present paper is to give a full scale account of Solynin's contin- 
uous symmctrization in the n-dimcnsional setting. Since the paper [27] is practically 
unknown to the experts (its English translation is often inadequate, actually it looks 
like a computer translation), we want to mention here the major innovations intro- 
duced in [27]. First of all, the polarization was used for the first time in [27] in the 
context of continuous symmetrizations. Then, an analog of the semigroup property 
was applied to prove some results about the continuous SC symmetrization. Later on, 
F. Brock [13] used a similar property as a part of the definition of his continuous sym- 
metrization. Uniqueness results were treated in [27] in all their generality. The latter 
leads, under certain conditions, to strict monotonicity of the domain characteristics 
under consideration as the functions of the parameter of symmetrization. Finally, the 
SC symmetrization was applied in [27] to prove local symmetry in some problems on 
Green's functions and harmonic measures. A similar approach to local symmetry in a 
more general context was also used in the papers [12] and [13]. 

This paper is organized as follows. Section 2 contains our basic notations. In particular, 
we introduce there necessary spaces of functions and classes of domains. In Section 3, 
we remind the reader of basic properties of the Steiner {k, n)-symmetrization and 
polarization. The exposition in Sections 2 and 3 follows the lines of our paper [14] joint 
with F. Brock. Sections 4-8 are devoted to geometric aspects of SC 1-symmetrization. 

In Section 9, we give an inductive definition of the continuous {k, n)-Steiner sym- 
metrization for any 2 < k < n. 

Sections 10 and 11, where we again follow the lines of the paper [14], contain our main 
applications. In Section 10, we show that many integral inequalities known for the 
Steiner symmetrization have their continuous counterparts for the continuous (k, n)- 
symmetrization as well. In Section 11, we give a similar treatment of the comparison 
theorems for solutions of some elliptic and parabolic PDE's. Many proofs in Sections 10 
and 11 related to the L^-classes and Sobolev classes are based on ideas suggested by 
F. Brock, when we worked on Sections 9 and 10 of our joint paper [14], and which he 
developed further in [13]. 

In the present paper we combined and extended the ideas and methods developed in 
[27], [28], and [14]. Preparing this article for publication, the author used his notes 
written in the Fall semester, 1995 during his stay at the Mathematisches Forschungsin- 
stitut Oberwolfach under the financial support of Volkswagen-Stiftung, RiP-program 
for Friedemann Brock and Alexander Solynin. Our intention at that time was to 
present in a joint paper our results for both SC fe-dimensional continuous symmetriza- 
tion and BC fe-dimensional continuous symmetrization. Since 1995, Brock's continuous 
symmetrization and its applications were already discussed in several publications. So, 
in this paper, we are concentrating on the Solynin's continuous symmetrization only. 
Although the original plan for this paper was changed, this work remains closely re- 
lated to the paper [14] joint with F. Brock, where such a possible continuation was 
referenced as "An approach to continuous symmetrization via polarization" . 

2. Preliminaries 

The following notations will be used throTighoTit the paper. Let be the Euclidean 
space, = {{xi, . . . , x„) G M" : a;^ > 0, 1 < i < n}. 
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For A C M", let A and dA denote the closure and the boundary of A, respectively. If 
A,B G M."" then A + B := {z : z = x + y, x & A, yG B} denotes the Minkowski sum 
of A and B. For x,y £ M" , by \x\ and {x,y) we denote the norm of x and the scalar 
product of x and y, respectively. Then H{a,n) and S(a, n) will denote the half-space 
{x G : {{x — a),n) > 0} and the hyperplane {a; G M" : {{x — a),n) = 0} defined by 
the point a G and the unit vector n G M"". For M C M", by C^{M) we denote the 
ra-dimensional Lebcsguc measure of M. By M-n-, ^n-, and Qn we denote the sets of all 
measurable, compact, and open subsets of M", respectively. Then M.n,h and Qnfi will 
denote collections of all bounded subsets of M-n and Qn- 

Generally, we treat measurable sets only in an a.e. sense, i.e. we write 

M = N if and only if £"(M A A^) = 0, 
MdN if and only if £"(M\Ar) = 0. 

By B^^^ {x{)) we denote the open ball in M" with radius r > centered at .tq and 
we write B^^ = 5^"^(0), = S^l^"). If yl C and £ > 0, then we denote by 
A^ := A+£B('^) the exterior parallel set of A. The Hausdorff distance between compact 
sets A and B is defined by 

d{A, B) := mf{e > : A C B„ B G A,}. 

It is well known that d is a metric on Tn- We define, by the metric d, the convergence 
of a sequence of sets Fi G Tn, z = 1, 2, . . ., to a set F G Tn by 

lim Fi = F if and only if d{Fi, F) — >■ as z ^ oo. 

If Q is an open set in M"" and p G [1, oo] then || • ||p denotes the usual norm in the space 
£^(0). For functions u G C(M"') we define the modulus of continuity by 

oJuiS) := sup{|'u(a;) - u{y)\ : \x - y\ < S}, S > 0. 

By W^'^{Q,) we denote the Sobolcv space of functions u G U'{Q) having generalized 
partial derivatives G LP{Q,), i = 1, . . . ,n, and we write 

n 

(2-1) := II^^IIp + XI W^^iWp 

1=1 

for the norm in this space. By Wq'^{CI) we denote the completion of the set of infinitely 

differentiable functions with compact support in denoted by C^(Jl), under the norm 
(2.1). Usually wc extend measurable functions u : $7 — t- by zero outside Q, so that 
W^'^i^l) C W'^'f'iW') in that sense. By C^'^in) we denote the space of Lipschitz 
functions with compact support in U. For any function space the lower subscript "+" 
denotes the corresponding subspace of nonnegative functions, e.g. L^{Q,), Wq^{Q,), 
etc. 

Let Sn denote the class of real measurable functions u satisfying 

> c}) < oo for all c > inf u. 

Here and in the following we use the following abbreviation: {u > c} = {x G : 
u{x) > c}. Note that the spaces L^(M"), Cq^^IM"), and the space W+'^(M") with 
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1 < p < oo are subspaces of Sn+- The space of measurable functions with bounded 
variation is denoted by BV{W^) and we write 



Recall also that if n G W^'^(M"), then ||Du||By = llVtxIli. Furthermore, if M is a 
Caccioppoli set in M", then ||Dx(M)||By is the perimeter of M in the sense of De 
Giorgi, see [32]. 

Finally, a function j : Mq" — )• is called a Young function if j is continuous and 
convex with j(0) = 0. 



3. Steiner symmetrization and polarization 



First we discuss some general properties of rearrangements. We remind the reader that 
a set transformation T (defined on Ain) is called a rearrangement if it is monotone 
and measure preserving, i.e. if T(A) C T(B) for all A and B such that A G B and 
£'^{T{A)) = £"(^) for every measurable set A. 

The class Sn introduced in the previous section is the natural class of functions for 
which a rearrangement can be defined. If T is a rearrangement and u is in Sn, then 

the relations 

(3.1) Tu{x) := ess sup {c > infu : x G T[{u > c})} inf Tu := inf n, 

define a function Tu on R". If u G 5„ is continuous, then "ess sup" in (3.1) can be 
replaced by "sup" . Clearly the function Tu is uniquely determined almost everywhere. 
Since T is measure preserving, 

£"(r({n > c})) = >C"({Tn > c}) for all c > inf n. 

Thus Tu £ Sn if u £ Sn- The mapping T : Sn ^ Sn constructed in this way is again 
called a rearrangement. The following non-expansivity lemma will be very useful in 
Sections 10 and 11, sec [14, Theorem 3.1]. 

Lemma 3.1. Let T be a rearrangement. Then for every Young function j, we have 

(3.2) / j {\Tu - Tv\) dx< j {\u - v\) dx for all u,v e Sn, 
whenever either one of the integrals in (3.2) converges. 

Sometimes we will say that two functions u,v E Sn are rearrangements of each other 
if inf u = inf u and jC."'{{u > c}) = >C"({v > c}) for all c > inf u. 

We will also use some additional properties of rearrangements. A set transformation 
T is called open or compact if T{A) is open or compact whenever A is of the same 
kind, respectively. We say that T is continuous from the inside if UiT{Gi) = T{UiGi) 
for every increasing sequence {Gi} C Q. Similarly we say that T is continuous from 
the outside if r]iT{Fi) = T{riiFi) for every decreasing sequence {F,} C T. 

Finally, a rearrangement T is called smoothing if T{Fr) D {T{F))r for every F E J- 
and r > 0. Smoothing rearrangements were introduced by Sarvas [25]. 

Let us now recall the definitions of the (A;, n)-Steiner symmetrizations (for further 
information see [30], [21], and [25]). 
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Definition 3.1. Every (n — k)- dimensional plane S C with 1 < k < n defines a 
{k,n)-Steiner symmetrization S as follows: 

For every x E T, let A(x) denote the k- dimensional plane through x and orthogonal to 
S. 

1) Let M e{J^n^ Qn) n Mn- If C^{M n K{x)) = 0, then S{M) n A(x) is empty or the 
point {x} according to whether Mfl A(x) is empty or nonempty. If C^{M r\A{x)) > 0, 
then 

(3 S) S(M) n A(x) = / ^''^'^^ ^ "^^^^ ^ "P"'''' 

^ ' ^ V 7 V y I B^(x)nA{x) if M is compact, 

where r > is defined by the condition C''(Brix) n A(x)) = C^{M n A(x)). 

2) Let M G M.n where M is neither open nor compact. Then the sets S{M) n A(x) 
are defined in an a.e. sense by either one of the equations in (3.3). 

From Definition 3.1 one deduces immediately that the {k, n)-Steiner symmetrization is 
a rearrangement which is continuous from the inside and from the outside. Note also 
that in case 2) Fubini's Theorem implies that the sets M fl K{x) are measurable with 
finite jC'^-measure for a.c. .t € E. 

The (n, n)-Steiner symmetrization is often called the Schwarz symmetrization or the 
symmetric decreasing rearrangement, and we will denote it by S*. 

For our purposes it will often be helpful to use a special coordinate system in M" = 
X R'^, where l<k<n, m = n — k and 

X = {xi, . . . , Xji) = {x , y), X = (xi, . . . , Xn—k)j y = {Xn—k+lj • • • j Xn), 

in which the plane E of symmetry becomes simply {y = 0}. If M G Mn, we introduce 
the "fc-slices" of M at x' by 

M(x') = {y G : {x',y) € M}, x' G R""*^. 

For instance, if x' G R"^''' with 1 < k < n, then Br^\x') will denote the /c-slice of 

the ball iJ^"^ at x' and not the ball in R*^ centered at x'. Let S*{M{x')) denote the 
Schwarz symmetrization of M{x'), taken in MJ^. Then (3.3) reads 

(3.4) SiM) := {x = {x',y) : y G S*{M{x')), x' G R"-'^}. 

If u G Sn, then we obtain from (3.3) that the (fc, n)-Steiner symmetrization Su of u is 
given by the relations 

(3.5) Su{x', y) = sup{c > inf u : x e S{{u{x', •) > c})}. 

(Here and in the following for simplicity {u{x', ■) > c} denotes {y G R*^ : u{x', y) > c}.) 

Let us mention again that the equations (3.4) and (3.5) have to be understood in the 
pointwise sense if and only if u is continuous. Note also that Su is "radially symmetric 
and decreasing in |y|", i.e. 

(3.6) Su{x',y) =Su{x',zi) >Su{x',Z2) if |y| = \zi\ < \z2\., 
where x' G W^~^ and y,zi,Z2 G M'^. 

Sometimes we will write S{M) = M* and Su = u* for the symmetrized objects. 

There is an approach due to Schwartz and Blaschke (see, for instance, [11]) reducing 
a A;-dimensional symmetrization to (k — l)-dimensional symmetrizations, see Theo- 
rem 4.32 in [25]. We will use a slightly refined version of this theorem. 
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Let be a (A;, n)-Steiner symmetrization in M" = x Mf' with the symmetry plane 
^ = {y = 0}- Let ^1 and if 2 be unit vectors orthogonal to S which form an angle 
77r, where 7 € (0, 1) is irrational. Let Si be the {k — 1, n)-Steiner symmetrization with 
the symmetry plane Sj defined by the plane S and the unit vector i = 1,2. 

For positive integer j and G J>j U Qn,b, let 

(3.7) nj = {S2 o Si)"'{n) if j = 2m is even, 

(3.8) = Sio {S2 o Si)"'{n) if j = 2m + 1 is odd. 
Here {S2 o >S'i)*^ is the identity transformation. 

Theorem 3.1. Let S, Si, and S2 be the symmetrizations defined above and let $7* = 
S{n). Then 

(3.9) lim d{Q.j, ^2*) = for every compact set O G Tn> 



(3.10) lim d{dilj,dri*) = for every bounded open set O, GOnb> 
and 

(3.11) lim £''{{nj{x') A n*{x')) = 
for every Q, G Tn U Qn,b o-nd every x' G M"~'^. 

For compact sets this theorem is a part of Theorem 4.32 in [25]. For bounded open sets 
the proof will be given in the Appendix. In Section 9, we will use the approximation 
scheme of Theorem 3.1 to give an inductive definition of our continuous (/c, n)-Steiner 
symmetrization. 

During the last decade we have seen increase of activity in the theory of symmetriza- 
tion, partly triggered by the paper [14], that is related to the polarization. This 
simplest rearrangement was introduced for sets by V. Wolontis [33] in 1952 who at- 
tributed some ideas of his paper to L. Ahlfors. Ahlfors himself used polarization in 
[2], where he introduced this transformation for functions. The term polarization was 
suggested by V. N. Dubinin [15]. 

Let S be a hyperplane in M" and let H be one of the open halfspaces into which 
is divided by S. Let an denote the reflection in S. We write x = (Th{x) for points 

X G and (th{u) = u{x) for all x G M" for functions u G 5. 

Definition 3.2. If u £ Sn, then its polarization Pu with the polarizer H is given by 

(o 1o^ p„r^^ / max{u(x),n(x)} ifxeH, 

{6.LZ) ru[x) .- I ^i^i^(^)^ ^(-^1 ifxeW\H. 

If M G A^ri) then the polarization P(M) is given by its characteristic function via 

(3.12) , i.e. 

(3.13) X(P(M)) := P(x(M)). 

In the case that u is continuous and M is open or closed, equations (3.12) and (3.13) 

have to be understood in the pointwise sense. 

Equations (3.12) and (3.13) can also be written in the following more precise form 

(3.14) P{M) = {{MUaH{M))nH)U{MnaH{M)), M e Mn, 
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and 

(3.15) Pu{x) = ess sup {c > miu : x G P({n > c})} , xeW, ue Sn. 

Of course, if u G »S„ is continuous, then "ess sup" in (3.15) can be replaced by "sup". 
Prom the representations (3.12)-(3.15) we see that the polarization P is an open and 
compact rearrangement which is continuous from the inside and from the outside. 

For the sake of simplicity, we will often use the subscript "jy" to denote any one of the 
polarized objects, i.e. we write uh and Mh for Pu and P(M), respectively. 

It is worth mentioning, that the polarization of a connected set is not necessarily 
connected and may contain one multiply connected component. 

There are three major approaches to polarization. The first one initiated in [33], [2], 
and [7] uses convolution type inequalities. In the most powerful and general form this 
approach culminated in Baernstein's fundamental work [6]. 

The second approach was introduced by V. N. Dubinin [15] who used the following 
representation of polarized functions. Let v{x) = u{x), w{x) = uh{x), x E H with 
xeH. Then 

uh{x) = {u{x) — v{x))-i^ and w{x) = u{x) — {u{x) — v{x))+. 

The latter under certain conditions leads to 

Vu{x) a.e. on {u > v} n H, 
Vv{x) a.e. on {u < v} Pi H, 

Vf (x) a.e. on {u > v} Ci H, 
Vu{x) a.e. on {u < v} Ci H, 

which easily leads to several Dirichlet type inequalities, see [14, Lemma 5.3]. 



Vuh{x) = 
Vw{x) = 



The third approach suggested by A. Solynin [28] and developed further in [14] rests 
on the direct application of the maximum principle to prove comparison theorems 
for solutions of two related boundary value problems for certain partial differential 
equations defined in a given domain O and in the polarized domain ^Iff. This approach 
will be used in Section 11. 



4. Continuous (l,n)-STEiNER symmetrization 

First we define a continuous transformation on R. For M e Mi and — oo < t < oo, 
the measuring function of M is defined by 

(4.1) mM{t)=jC\{-oo,t)nM). 

This definition shows that mM{t) is nondecreasing and Lipschitz continuous with con- 
stant 1: 

(4.2) 0<mM{t2) -niMih) <t2~h for alHi < 
This implies that, for every i G M, the equation 

(4.3) y-{l/2)mM{y)=t 

has a unique solution y = yM{t) G [t, oo). The function y = yM{t), called the separating 
function of M, will play an important role in this study. Two basic properties of yuit) 
given by the following lemma are immediate consequences of the above definitions and 
inequalities (4.2). 
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Lemma 4.1. (a) IfMcN, then ynit) < 2/iv(i) for all teR. 
(b) Ifti < t2, then 

(4.4) t2-tx< yM{t2) - VMih) < 2{t2 - ti). 
For t G M, the r*-transformatioii of M is defined as 

(4.5) M* = (yMit) - mMit),yMit)) U (M n [yM(t), oo)) if M is open, 

(4.6) M* = [yM{t) — mM{t),yM{t)] U (M n [yMit), oo)) if M is compact. 

If M G Mn but is neither open or compact then M* is defined in the a.e. sense by 
either one of the equations (4.5) or (4.6). 

Definition 4.1. The family of mappings T* : Mi Mi, t eR, defined by T^{M) = 
M* is called the continuous symmetrization on R. 

Now we turn to R" = R""^ x M. For Q G Mn and x' G R"~\ let n{x') = {y G M : 
{x', y) G $1} be the 1-sHce of J7 at x' . If 17 G is open or compact then fi' will 
denote the orthogonal projection of Q, onto M"~^. Otherwise we put fi' = {x' G M"~^ : 
/^^(r2(a;')) / 0}. The measuring function of $7 is defined by 

™o(x',<)^ 1^:^"^ 

One can easily show that m^{x' , t) is lower semicontinuous in x' if J7 is open and it is 
upper semicontinuous in x' if is compact. 

Definition 4.2. T/te function ya : R""^ x R ^ M de/inec? 6y 

is called the separating function and the graph Ffi(t) = {{x' , ya{x' , t)) : x' G M"~^} is 
called the frontier of symmetrization of Q at t. 

To simplify notation we will skip the symbol of the set if its meaning is clear from the 
context. Thus we often write m{x',t), y{x',t), etc. instead of mQ{x',t), yQ{x',t), etc. 

Lemma 4.2. For a fixed t G M and fl G Mn, the separating function y{x' , t) is lower 
semicontinuous on M"~^ if J7 is open and it is upper semicontinuous on R""-*^ if O is 
compact. 

Proof. Let be open and x'q G R"~^. If m{x'Q,t) = 0, then y{x',t) > t = y{xQ,t) for 
all x' and the lower semi-continuity follows. 

Assume that y{x',t) is not lower semicontinuous at x'q such that m(x'Q,t) > 0. Then 
for some 6 > and some sequence — t- Xq, 

(4.9) yixlt)<yix'o,t)-S, k = l,2,.... 
From (4.3), (4.2), and (4.9) we obtain 

(4.10) m(xfc, yix'o,t) -6)- 2{y{x'k,t) - t) = m(4, y(xo, t) - 5) - m{x'k,y{x'k,t)) 

<y{xo,t) -y{x'k,t) - (5. 

This combined with (4.9) gives 

(4.11) limsupm{x'^,yix'o,t)-d) < 2{y{x'o,t) - t - 6) . 
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Since J7 is open the function m(x', t) is lower semicontinuous in x'. Hence 

(4.12) limmfm{x'k,y{xQ,t) - 6) > m(xo, y(xo, t) - 5). 

Using (4.2) and (4.3) we obtain 

(4.13) m(x^, y(.T'o, t) - 5) > m(x^, y{x'o,t)) -6 = 2(y(4, t) - t) - 5 
Now (4.12) and (4.13) yield the inequality 

liminf m(a;'fc,y(a;o,t) - S) > 2{y{xQ,t) - t) - 6, 

k—^oo 

which contradicts (4.11). This proves Lemma 4.2 for the case of open sets. If O is 
compact the proof is similar and is left to the reader. □ 

Now we are ready to define what the 1-dimensional continuous symmetrization is. 
Definition 4.3. A family of set transformations T* : 7W„ — )■ Ain> t defined by 

(4.14) T\n) = := {{x',y) : x' e n' , ye n\x')} 

will be called a continuous 1- symmetrization or SC 1-symmetrization. Any single trans- 
formation of this family will be called a -transformation. For t = —oo, we define 
T~°° to be the identity transformation. 

The set fi* itself will be called the T* -transformation ofil or the partial symmetrization 
of ^ with respect to the plane {y = t}. Sometimes we will refer to this plane {y = t} 
as the moving plane of symmetrization. Accordingly, the parameter t will be called the 
height of the moving plane. 

The frontier of symmetrization F^^t) divides M" into two parts 

(4.15) H+{t) = {{x',y) : y > y{x',t)} and H_ = {{x',y) : y < y{x',t)} 
called the upper and lower subspaces of symmetrization, respectively. Let 

(4.16) n-{t) = nnH_{t), n+{t) = n\H_ {t) if n is open 

and 

(4.17) n+{t) = nnH+{t), n_{t) = n\H+{t) if $7 is compact. 

If 0, is neither open or compact then Jl_(f) and ^+{t) arc defined a.c. by (4.17). 

In Lemma 4.3 below we list some useful properties of SC symmetrization, which follow 
directly from the above definitions. By ^'L{t) we denote the (1, n)-Steiner symmetriza- 
tion of n_(t) with respect to {y = t}. 

Lemma 4.3. (a) = ^}*_{t) U f7+(t) for allt and every Q £ Mn- 

(b) C^{n\x')) = £i(17(x')) for all x' G M""^ and = £"(0). 

(c) If O, is open, then Q-{t), 01 (i), and are open sets and 

n*_it)n{y>t} = H_{t)n{y>t}. 

(d) If Q is compact, then 0_(t), Q'^{t), and f2* are compact sets and 
n*_it) n{y >t} = {y >t}\H+it). 

We note that Lemma 4.3(a) gives an equivalent definition of and Lemma 4.3(b) 
shows that the r*-transformation preserves measures. 

Now we will show that SC symmetrization possesses all basic geometric properties of 
classical symmetrizations. 
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Lemma 4.4. For a fixed t E 'R, the -transformation is an open, compact, and 
smoothing rearrangement, which is continuous from the inside and from the outside. 

Proof Lemma 4.1(a) and Definition 4.3 imply tliat T\M) C T\N) if M C iV 
and therefore the r*-transformation is monotone. Since, by Lemma 4.3(b), the T*- 
transformation preserves the measure of sets, it follows that the r*-transformation is 
a rearrangement. 

From Definition 4.3 and Lemma 4.3(c) and (d), one can easily verify that the T*- 
transformation is continuous from the inside and from the outside. 

To prove that the T*-transformation is smoothing, wc fix a compact set K and e > 0. 
Since, by Lemma 4.3(a), = Kl{t) U K+{t), we have 

(4.18) if* + £SW = (ii:l(t) +eBH)u (K^ + eSH). 

Using Lemma 4.3(d) and the smoothing property of the Steiner symmetrization we 
obtain 

(4.19) T\K + eSW) D T\K_ {t) + eSH) = {K^ {t) + e^H)* d Kl (t) + e^H, 

where (•)* denotes the Steiner symmetrization with respect to {y = t}. Since K^{t) 
remains unchanged under the T*-transformation of K one can easily see that 

(4.20) T\K + eSW) D T\K+{t) + eSH) = K+{t) + eB^. 
Now (4.18) - (4.20) yield: 

T\K + eSW) D T\K) + eB^, 
which shows that the r*-transformation is smoothing. 

It was shown by Sarvas, see Lemmas 3.2 and 3.3 in [14], that the properties established 
above imply that the r*-transformation is open and compact. The proof is complete. 

□ 

Since for any fixed t G M, the r*-transformation is a rearrangement of sets there is a 
standard way to define a corresponding transformation of functions: 

Definition 4.4. SC 1- symmetrization of functions: Let u G Sn- Then the family of 

functions u^, t G M, defined for x £ by 

,,t(^^ ._ / esssup {c> infix : xe T\{u > c})} if x e Uc>inf«r*({^ > 4) 
^ I inf« ^f x^UoinfuT'auyc}) 

is called the SC 1- symmetrization of u. 



5. Continuity properties of SC symmetrization 

Now we prove that the r*-transformation depends continuously on the parameter t, 
which justifies the use of the word "continuous" in the name of this transformation. 
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Lemma 5.1. Let t^, k = 1,2, . . . be an increasing sequence such that t^ t' < oo and 
let = J^*' and Qk = • Then 

(5.1) d{dnk,dn') ^0 as A; ^ oo 
if J7 is an open bounded set and 

(5.2) d(Ofe,J^')^0 as A;^oo 
if ^ is a compact set. 

Proof. We will prove the lemma for open sets. The proof for compact sets follows the 
same lines, it is easier and is left to the reader. 

If (5.1) does not hold, then there exist subsequences of the sequences tk and Clk, which 
we still call tk and Clk, such that 

(5.3) d{dnk,dO,')>e, A; = 1,2,... 

for some e > 0. Since Q is bounded equation (5.3) implies that there arc subsequences 
(again denoted by t/. and ilfc) and a sequence of points G M"' with — >■ xq G ffi** 
such that one of the following two conditions is satisfied: 

(a) Xfc G dW and dist(xfc, d^lk) ^ £ for all A; = 0, 1, 2, . . . 

(b) Xk G dQk for A; = 1, 2, . . . and dist(xfc, dQ') > e for A; = 0, 1, 2, . . . 

The case (a) contradicts Lemma 5.2 below, which characterizes the stability property 

of the boundary of O*. 

Now we consider the case (b). Let 

Xk = {x'k,yk) {x'o,yo) = xq. 

Then 

xk£d{n*_{tk))ud{n+{tk)). 

Taking a subsequence if necessary we may restrict ourselves to two cases: 

(1) In the first case we assume that Xk G dCl^{tk) for all A; = 1,2, Since Xk fife 

we have Xk ri+(tfe). Now if Xk G then Xk G Q-{tk). Since r2_(tfc) is open and 
r2_(tfc) n 0,^{tk) = we conclude that xj. dO,^{tk), which contradicts the above 
assumption. Thus in the case under consideration Xfe and therefore Xk G dO, for 
all A; = 1, 2, Taking the limit we get 

(5.4) xo G dn. 
Since Xfe G dQ,+ {tk) using (4.4) we obtain 

Vk > y{x'k,tk) > y{x'k,t') - 2(t' - tk). 
Passing to the limit we get 

(5.5) yo>y{x'Q,t') 

since y{x',t) is lower semicontinuous in x'. Equations (5.4) and (5.5) imply that 

(5.6) Xo n'. 

Now we will investigate possible dispositions of i/q, y(xg,i'), and t'. 

(i) If yo = y{xQ,t') > t' , then xo G dO,' contradicting the assumption (b). 
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(ii) The second possibility is 

yo > y{x'Q,t') > t'. 

It follows from the assumption (b) and (5.6) that there exists a ball (xq) with a 
small radius £i > such that 

(5.7) B^{xo)c{R^\n')nH+{t'). 

By (5.4), there exists a point x G Qr\Bi\\xo). Since Be^\xQ) C H^{t') we must have 
X G ri'. But this contradicts (5.7) and therefore contradicts (b). 

(iii) Let yo = t'. The assumption (b) and (5.6) imply that there is a ball Bs^\xo) such 
that Bi'l\xo) C M" \ n'. Then it follows from the definition of n^' that Bt\xo) C 
M" \ n, which contradicts (5.4). This finishes the proof of the lemma in the case (1) 
under the assumption (b). 

(2) In the second case we assume that 

(5.8) Xk e d{n*_{tk)) \ dn+{tk) for all A; = 1, 2, . . . 

As in the case (1) we consider possible dispositions of yo, y{xQ,t'), and t'. 

(i) If yo < t', then y^ < tk for all sufficiently large k. Since Xk G 5(01 (t^)) and 
since the open interval between the points (x^, 2tk — y(x'j^,tk)) and (x^, y(xj^, tjfc)) is in 
^*-{tk) we have 

(5.9) yu < 2tk - y{xi,tk) < 2t' - yix^t') 

for all k large enough. The second inequality here follows from (4.4). The latter 
inequality implies that X}- and therefore xq is not an inner point of 

Let us show that xq cannot be an outer point of fi', too. Since Xk G dQ^{tk) and 
Xk ~^ ^0 there is a sequence of points Xk = {x'^,yk) £ ^*-{tk) such that Xk — >■ xq- Then 
we have 2tk — y{x'i^,tk) < yk- Therefore using (4.4) we obtain 

2t' - y(4, t') < 2tk - y{x'k,tk) + {f - tk) < a' - tk) + Vk- 

This yields (x^, {t' — tk) + yt) G Since xq ^ fi' taking the limit we get xq G di}', 
which contradicts the assumption (b). 

(ii) Let yo > t'. Then y{x'j^,tk) < yk for all k large enough. Therefore by (4.4), 

2/(x'fc, t') < 2{t' - tk) + y{x'k,tk) < 2{t' - tk) + yk. 

Passing to the limit and using the semi-continuity property of the separating function 
we obtain 

(5.10) y{x'o,t') <yo. 

The case of equality y{x'Q,t') = yo (> t') leads to the inclusion xo G dO,', which 
contradicts the assumption (b). Indeed, if xq ^ d^', then xq G Cl'. Therefore Xfe G O 

for all STifficiently large k. Since Xk G 3J71(tfc) and Xk G 0,, it follows that Xk G J7+(tfe). 
Therefore Xk G dO,jf-{tk) for all k large enough, which contradicts (5.8). 

Now let y(xQ,t') < yo. If xq ^l', then it follows from the assumption (b) that there 

15 a bah B^fixo) with some small radius £2 > such that 

B^{xo)c{R^\n')nH+{t'). 
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Figure 1. r*-transformation of sets. 



Then ^^"^(xo) nn = $. Since B^'^'ixo) C H+{tk) we must have B^'^'ixo) n = for 
all = 1, 2, . . . The latter is impossible since Xk G dilk and Xk — )■ xq. 

If xq G ri' and y{xQ,t') < yo, then xq G O. Hence, G for all sufficiently large k. 
Since Xk G 5r21(ifc) this implies that Xk G Jlfe contradicting the assumption (b) if k is 
large enough. This completes the proof of the lemma in the case yo > t'. 

(iii) Let yo = t'. If xo ^ ^' then by the assumption (b), i?!"-* (xo)nfi' = 0. This implies 

that Bi"'\xo) n Q.'^{tk) = for all A; = 1, 2, . . . The latter contradicts the assumptions 
Xk G dil*_{tk) and x^ — >■ xq- 

Let Xo G n'. Then by the assumption (b), Bs^\xo) C 01 (t'). This together with the 
inequality (4.4) implies that B^^^{xo) C ^*_{tk) for all A; large enough, again contra- 
dicting the assumption (b). The proof of the lemma for open sets is now complete. 



Figure 3 shows the results of continuous symmetrization of a domain Q, for a few 
values of t. Comparing domains in Figure 3c and Figure 3d, the reader may see that 
the r*-transformation is not continuous from the right in the Hausdorff metric. 

Now we will prove the lemma characterizing a stability property of the boundary t^fi*. 
Lemma 5.2. Let — oo < s < t < oo. Then 



if Q is a compact. 

Proof. Once more we give a proof for open sets. The easier case of compact sets is left 
to the reader. 

Suppose that Q is open and that (5.11) is not satisfied. Then there exists a point 
Xo G dO,^ such that 



By Lemma 4.3(c), the set Q-{t) is open. By the definitions of Q_(i) and Q+{t), we 
have n-{t) n n+[t) = 0. Therefore (5.14) implies that xo ^-{t). 



□ 




dn* C + {t- s)B(^) 



c + (i - s)B(") 



(5.13) dist{xo,dn^) > (t-s). 

By Lemma 4.3(a), we have = Q,*_{t) U 0+(t). Hence, 

dn^ c dn*_{t)udn+{t). 



As in the proof of Lemma 5.1 we consider two cases. 

(1) In the first case we assume that 

(5.14) xo G dn+{t). 
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Since ^+{t) C CI we must have xq G ^- If xq G $7, we conclude that Xq G ^+{t) 
and therefore xq G ^2*. Since by Lemma 4.3(c), $7* is open, the latter contradicts the 
assumption xq G dfl*^. Therefore we must have 

(5.15) Xq G dn. 

Since fi+(t) C J^+(s) C Jl, equations (5.14) and (5.15) imply 

(5.16) XQ G dn+{s). 

Hence, xq G O*. This implies that xq G 50*. Indeed, let xq G fi''. Since we 
conclude that xq f2+(s). Therefore in the case under consideration we must have 

(5.17) Xq G n*_(s). 

By Lemma 4.3(c), the set WL{s) is open and it follows from the definitions that 
$71 (s) n J7_|_(s) = 0. Therefore (5.17) contradicts (5.16) and we must have xq G d^l^ 
contradicting (5.13). 

(2) In the second case we assume that xq = (xqjI/q) G dQ'L{t) \ ^0,-^-{t). Now we 
consider two subcases. 

(i) Suppose that uq < t. Since xq G we have 

yo<'2t- y{x'Q,t). 

This together with (4.4) implies 

yo- {t- s) <2s - y(xo,s). 

This inequality yields 

(5.18) x:={x'^,yQ-{t-s))^Vt'. 

Since xq G dQ.*_{t) then for each e > there exists a point x = {x',y) G 0!l(i) such 
that 

(5.19) \x — xq\ < £. 

Let Uq < s. Then m{x',y) > and therefore m{x',s) > if x is close enough to xq. 
Therefore the interval 

((£', y) : 2s- y{x', s) < y < y{x', s)) 
is nonempty and belongs to ^2*. 

Using (4.4) and the relation x G 01 (t) we conclude that 

2s - y{x, s) <2t- y{x', t) < y < s 

if e > in (5.19) is small enough. Then we have x G J7^. Therefore the closed interval 
[x,x] contains a point z G dO,^. Hence, 

\z — xq\ < max{|xo — x|, |xo — x|} = (i — s) 

if £ > in (5.19) is <t — s, which contradicts inequality (5.13). 

Let yQ = s and let y < s. Then m{x',s) > and (x',s) G O*. The closed interval 
[xo,(x',,s)] contains a point z G d^^. Hence, \xq — z\ < e, which again contradicts 
(5.13) ii£<t-s. 

Let yQ = s, y > s. If m(x', s) > 0, then we argue as in the previous case. 
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Let m(x', s) = 0. Since m{x', y) > then there exists a point z = {x', C) ^ ^ such that 
s < C ^ y- But in this case z G and \z — xo| < |£ — xo\ < e. Since e > is arbitrary 
small this easily leads to a contradiction to (5.13). 

Let s < yo <t. If m{x', s) > 0, then [x', s) G Jl* and 

ko - {x,s)\ < \xo - {x'q,s)\ + \ ix'Q,s) - {x',s)\ < {t- s) + e. 

Since e > can be chosen arbitrary small the latter inequality contradicts (5.13). 

Let m(.x', ,s) = 0. Since m{x',y) > then there exists a point z = {£', Q G Q, such that 
s < C < y. Then z G O*. In addition, 

\z — xo\ < {t — s) + e. 

Since e > can be chosen arbitrary small the latter inequality again contradicts (5.13). 

(ii) Let yo > t- Since xq fl* we have yo > y{xQ,t) > y{x'Q,s). This implies that 
xo ^ Q,^. Since xq G 5r21(t) we conclude that for every e > there exists a point 
X = {x',y) G r21(t) such that 

(5.20) \x - xq\ < e. 

If e > is small enough we have t < y < y{x' ,t). 

If y{x',s) > y, then x elF. Therefore (5.20) contradicts (5.13). 

Now we consider the case y{x',s) < y. Let / := {[x',y{x',t)), (x' ,y{x' , s)) and let 
Qj = I n Q. It follows from (4.3) that the linear measure of Qj equals 2{y{x',t) — 
y(x', s)) — 2(t — s). Since x G / we have 

dist(x,Sl/) < C^{I)-C\ni). 

This inequality together with (4.4) implies 

dist(f , 17/) < 2{t - s)- {y{x', t) - y{x', s)) < t - s. 

Since 0/ C O'' there exists a point z € fl^ such that jx — z| < t — s. Finally, we have 

\xo — z\ < \xo — x\ + \x — z\ < (t — s) + s. 

Since e > can be chosen arbitrary small the latter inequality contradicts (5.13). The 
proof of the lemma for open sets is complete. □ 

6. Stability and semigroup properties of T*-transformation 

First we prove a lemma characterizing stability with respect to polarization, cf. [27, 
Theorem 3]. 

Lemma 6.1. For s G M, let Pg denote the polarization with the polarizer Hg = 
{{x',y) G M" : y > s}. If ti < t2 and s < (l/2)(ii +t2), then 

(6.1) r*2 o o T*i = r*2 onMn- 

Proof. Let G be an open set and let D = Pg ($7*^ ) . We have to show that 

(6.2) D^^ = n^\ 

By Lemma 4.3(a), 

(6.3) D^^ = D*_{t2) u D+{t2), n^' = n*_{t2) u n+{t2). 
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Since the r*-transformation and polarization both preserve measure in 1-shces, it fol- 
lows from (6.3) that (6.2) holds true if and only if 

(6.4) D+{t2) = n+{t2). 

Since $1*1 n {{x',y) : y < 2t\ — y^i{x\ti)} = 0, it follows from the definition of 
polarization and r*-transformation that 

(6.5) P.(n*i)n {{x',y): y > 2{s - h) + yn{x' ,h)} 

= n'^n {{x', y):y>2{s- h) + yn{x', h)} 
= n n{{x',y) : y>2{s-h) + yn{x',h)}. 

Since s < (l/2)(ti +t2), (4.4) implies 

(6.6) yn{x', ta) > yn{x', h) + 2{s - h). 
Now from (6.5) and (6.6) wc conclude that 

Ps{n'^) n {{x',y) : y > yn{x',t2)} = nn{{x',y) : y > yn{x',t2)}, 
which implies (6.4). 

For open sets the proof is complete. If Q is not open the proof follows the same lines. 

□ 

Lemma 6.1 and Definition 4.4 immediately imply: 

Corollary 6.1. Let u{x) G Sn, h < t2, and let s < (l/2)(ii + 12). Then 

In the particular case s = ti, Lemma 6.1 and its corollary show that our r*-transformation 
possesses the following "presemigroup property" , which was first mentioned in [27] . A 
similar property was used in [13] to define the BC symmetrization. 

Corollary 6.2. Let T* denote a -transformation on Mn 

or on Sji- Then 

(6.7) o r*i = for all ti<t2. 

To justify the name "presemigroup property", we consider the transformation T* = 
St*' o r*, where t > and St* denotes a continuous shift of t units in the direction of 
the negative y-axis. Then one can easily see that equation (6.7) is equivalent to the 
following familia semigroup property of T*: 

ft+s ^ft^fs alH > 0, s > 0. 

In Section 8, a similar composite transformation will be used to replace the parameter 
t, the range of which is — oo < t < oo, by a parameter r with the standard range 
< r < 1. 

Now we prove a criterion for Q, and u to possess a partial symmetry . 

Lemma 6.2. (a) Ifil G Mn is open or compact, then Q = il* if and only if Ps{'^) = 
for all s < t. 

(b) If u & Sn, then u = u* if and only if uh, = u for all s < t. 
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Proof, (a) We work with an open set The necessary part is obvious: If O = Q*, 
then Lemma 4.3(a) shows that Ps{^) = ^ for all s <t. 

To prove the sufficient part, we assume that Ps($l) = $7 for all s < t but Q 7^ il*. Since 
n+{t) = we conclude that / n*_it). Then by Lemma 6.3 in [14], there 

is a polarizer Hsq = {y > sq} with sq < t such that 

(6.8) Ps,{n-{t))^n_it). 

Then there is xq = (xq, yo) with yo < ■^o such that 

(6.9) xo £ ^-{t) and x := (x'q, 2so - yo) ^-(t) ■ 

If 2so — yo < y{x'o,t^, then (6.9) contradicts the assumption Ps(Jl) = for s = so < t. 

Let 2so — yo ^ y{x'o-,i)- Since Ps(0) = for all s < t and G f^, it follows that the 
closed interval {{x'Q,y) : yo < y < 2t — sq} is in CI. Then 

mn{x'o, y{xQ, t)) > y{xQ, t)-yo> 2{y{xQ,t) - sq) > 2(^(4, t) - t) 

contradicting the equation (4.3). The proof of Lemma 6.2(a) for open sets is complete. 

If ri is not open the proof is left to the reader. 

The proof of Lemma 6.2(b) easily follows from Lemma 6.2(a) and Definition 4.4. □ 
Our next result will be used in the proof of Theorem 11.3 in Section 11. 

Corollary 6.3. Let be a bounded domain in and let f G L^(Q), / > on O. // 
for some T G M, is not a translation of f in the direction of y-axis, then we can 
find a polarizer Ht^ = {y > to} with to < T such that ^fHt^) = f^j f 7^ fHt^j o.'^d 

In particular, if is not a translation of in the direction of y-axis, then we can 
find a polarizer Ht^ = {y > to} with to < T such that {'^Ht )^ = j ^ / ^-ff* ; "^^^ 

Proof. Let ti = inf t, where the infimum is taken over all t G M such that ^-\-{t) = 0. 
Then for all i > ^i, /* is a translation of /*^. Therefore, proving this corollary, we may 
assume that T <t\. In this case, f^ is not a translation of / if and only if /-^ 7^ /. 

Let to, = supt, where the supremum is taken over all t G M such that Prif) = / for all 
T < t. Then, = / by Lemma 6.2. Since f'^ 7^ /, it follows from the same lemma 
that ti^ < T. Combining this with Corollary 6.1, we obtain that 

Since tfi < T, /*" = /, and Q+{tfi) ^ 0, we must have (Tif^(J7) 7^ ^h^j ^'iid therefore 
(^Hsif) 7^ fHs7 foi' ^-ll "5 sufficiently close to tQ. Therefore, every polarizer Hg with 
t(n < s < {l/2)(tfi +T) such that fn^ / / will satisfy the requirements of the corollary. 

By the definition of to, there is a sequence {sfcj^i, which converges to ta from the 

right, such that ///^ 7^ / for all k. Choosing A; sufficiently large such that Sk < 
(l/2)(ff2+T), we obtain the desired polarizer Hfg = Hs^- This proves the first assertion 
of the corollary. Since O = {/ > 0} the second assertion is a particular case of the 
first one. □ 

Lemma 6.3. Let Q G Mn o-nd let (•)* denote the {l,n)-Steiner symmetrization with 
respect to the plane {y = t}. If T\n) = andneTUGb then T\n^) = {U^)* for 
every e > 0. 
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Proof. Assume that CI G Gb- For a fixed e > 0, let ye{x',t) and me^x' ,y) denote, 
respectively, the separating function and measuring function of fi^. For x' G (f^e)', let 

y,(x') = sup{y : {x' ,y) eT\^,)}. 

Then ye{x') > ye{x',t). If ye{x') = ye{x',t) for all x' G (J^e)', the conclusion of the 
lemma is immediate. 

Suppose that 

for some Xq G (fie)'. Then (4.3) implies 

(6.10) m,(4,2/,(4)) < 2(y,(x(,) - t). 

Now we fix (5 > sufficiently small. Then we can find two points x = (xo,y) and 
X = {x',y) satisfying the following conditions: 

(6.11) xeT\n,) and y^ix'^) - 5 < y < y,{x'o), 

(6.12) X G and |x — x| < e. 
Since T*(J7) = fi* and x G $7 we have: 

(6.13) mix',y)>2{y-t). 

Using (6.12) and (6.13), after an elementary geometric argument we obtain the in- 
equality 

me{xQ,y) > 2{y - t), 
which together with (6.11) implies 

me{xQ,y^{xo)) > me{x'o,y) > 2{y^{x'o) -t)- 25. 

Since (5 > can be chosen arbitrary small, the latter inequality contradicts (6.10). 
This proves the lemma for open sets. If f2 is a compact set the proof follows the same 
lines. □ 



7. Approximation by polarizations 



A compact set X C is said to be simple if it can be decomposed into a finite number 
of blocks Ri = R{a^,V) such that Rif^Rj = ^\ii ^ j. Here R{a, b) denotes the closure 
of an open block 

R{a, b) := {x G M" : b^ < x^ < bk + ak,l < k < n}, 
where a = (ai, . . . , a„) G M" and 6 = . . . , 6„) G M". 

If i^T is a simple compact set in M" = M"~^ x M, n > 2, then its projection K' is a 
simple compact set in R""^. Let K' = U^^R[ be a block decomposition of K'. 

A collection of open sets Cu{K) := {Cj = R'j^x M}^ will be called a cubicle structure 
of K. If Ci is a cubicle in Cu{K), then Ci f\ K = R\ y. K^, where is a simple 
compact set in R. Therefore d H K consists of a finite number of disjoint blocks 
Kij = R'^ X [aijjPij], 1 < j < ji- Wc always enumerate these blocks such that 

Oil < Pii < ai2 < • • • < ctij^ < f3ij^, 1 <i < N. 

Thus every simple compact set K admits a block decomposition of the form 

AT 

K = \jR^iXK' = (jKij. 
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In this section we show that the r*-transformation of sets and functions can be approx- 
imated by a sequences of polarizations. This approach was first used by V. Wolontis 
[33]. The method was developed further by V. N. Dubinin [16], [17]. So we will call this 
method the Wolontis-Dubinin approach. In the context of continuous symmetrization 
the Wolontis-Dubinin approach was first used in [27]. 

Lemma 7.1. For every t G M and every collection of simple compact sets Ki, . . . , 
in M" there exist a finite number of polarizations Pk, ^ < k < N with polarizers 
{y > Vk} such that 

(7.1) yi <y2 < ■■■ <yN <t 

and 

(7.2) OtiPk{Ki) = Kl for all 1 < i < m. 

Proof. (1) Let t G M be fixed. First we consider the case n = 1. Then the block 
decomposition of Ki consists of a finite number of disjoint segments: 

ji 

(7.3) ^i = [J 

where 

(7.4) an < bn < aj2 < &i2 < • • • < aij^ < bij^, ji>l, 1 <i <m. 

For every index i we define two numbers: 

(7.5) ai = (1/2) {t + (1/2) {an + bn)) , 
and 

(1/2) {an + ai2) if > 1 



(7.6) A 



t otherwise. 



Let A[^^ denote the set of all numbers and all numbers /3j that are less than t. It 
is easy to verify that T* {{Ki)) = Ki for some i if and only if ctj > t. Therefore if 
Af^^ = 0, then ai>t for all i and there is nothing to prove. 

If A[°^ / 0, then we set 



yi := min{7 : 7 G A^f^}. 



Now we consider a polarization Pi with the polarizer {y > yi}. Let K^'^^ = Pi{Ki), 
1 < i < m. For each i, the compact set K^'^^ satisfies the following conditions: 

(a) is a simple compact set whose block decomposition K^^^ = 
[jf=i[C'ij \ ^ij^] consists of j^^^ disjoint segments, where j^^^ < j^. 

(b) (^i^^)^ (i) = {Ki)+ {t) and therefore r*(i^f ^) = T\Ki) for all 1 < i < m. 

Now starting with simple compact sets K^^^^ , 1 < i < m, we define a set A^^^ of elements 

Oj-^^ < t and pI^^ < t in the same way as .4^°^ was defined for Ki, see formulas (7.3)- 

(7.6). As we noticed above, if A^^^ = 0, then iff ^ = T*(Kf ^) = T\Ki), 1 < i < m, 
where the second equality follows from the condition (b) . In this case our construction 
is finished. 
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If ^j^^ / 0, we continue our construction further to get simple compact sets K^'^\ 

K^^\ ... , 1 < i < m. Let A\''\ af'\ lif^\ and yk+i correspond xf^^ in the process of 

this construction. One can easily see that in each step of our construction xf''^ , a[''^ , 
and Uk+i satisfy the following conditions: 

(i) If for some k>l, ^'^^ ^ 0, then yi < y2 < ■ ■ ■ < Vk < t. 

(ii) If for some A; > 1 and some s, 1 < s < m, yk = ai''~^\ then #A\''^ < 

(k—l) 

ij^Al — 1, where # denotes the cardinality of the corresponding set and 

(iii) If yk = (3^ '^^ for some s, 1 < s < m, then < — 1, where jf^ denotes 
the number of disjoint segments in Kf^ if / > 1 and j^^^ = ji. 

Since the total number of segments constituting Ki, . . . ,Km is finite and i^At < oo 
the conditions (ii) and (iii) show that the above construction of compact sets K^'^^ 
terminates after a finite number of steps. Thus there is an integer N > such that 
A[^^ = 0. Since in each step of our construction the conditions (a) and (b) are satisfied 
we have 

KiN) ^ (^^(iv)) ^ (^^(iV-D) =... = T\K,), l<i<m, 
which proves the lemma in the case under consideration. 

(2) Now we consider the case n > 2. Since Ki is a simple compact set in M" it can be 
decomposed as follows 

where R'ij are closed blocks in R"~^ such that R'^^j Ci R'-i = $ if j I and are simple 
compact sets in R. 

By part (1) of this proof, for every t G M there are polarizations P^, 1 < A; < A^, in R 
with the polarizers {y > yk} such that 

yi < y2 < ■ ■ ■ < yN < t 

and 

O^^^PkiKj) = T\Ki) for all 1 < j < ii and 1 < i < m. 

Considering Pk, 1 < k < N , as polarizations in R" with the corresponding polarizers 
{y > yfc} we obtain 

ji 

Ok=iPk{Ki) = (j R'ij X (t*(k/)) =T'{Ki), l<i< m. 

The proof of the lemma is complete. □ 

Corollary 7.1. Let t eM. and let D, Q be bounded open sets such that D C CI. Then 
there is a finite number of polarizations Pk, ^ < k < N, with polarizers {y > yk}, 
where yi < y2 < ■ ■ ■ < Vn ^ t, such that 

DNCn\ where Dn = 0^=iPk{D). 
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In addition, 

dist(5r>, dn) < dist{dDN, dQ^), where Dn = Of=i^fcP)- 

Proof. Let po = dist(t?Z), dfl). Then po > and for every p, < p < po, there exists a 
simple compact set K = K{p) such that 

{D)p cK cn. 

By Lemma 7.1, we can find a finite number of polarizations Pfe, 1 < A; < iV such that 

Since the T*-transformation and polarization both are monotone and smoothing trans- 
formations we have 

(7.7) D = D OfcLiPfe((^)p) ^ {Dn)p. 

Since (7.7) holds true for every p, < p < pQ, the corollary follows. □ 
Definition 7.1. A function f E Sn is said to be simple if f can be represented as 

(7.8) f = s(^-no + f2x{K^)^ 

with some £ > 0, no G N and some simple compact sets Ki, 1 < i < m, such that 
D ••• D Km- 

We note that simple functions are dense in L^(M") for every 1 < p < oo. The following 
lemma shows that the T*-transformation of a simple function can be reduced to a finite 
sequence of polarizations. 

Lemma 7.2. Let f : M" — )■ M 6e a simple function. Then for every t G M there are 
polarizations Pk with polarizers {y > Uk}, I < k < N such that 

(7.9) yi<y2< ■■■ <yN <t, 
and 

(7.10) OLiPkif) = f- 

Proof. Since / is simple it can be represented in the form (7.8). By Lemma 7.1, we 
can find polarizations Pk, I < k < N, with the polarizers {y > yk} satisfying (7.9) 
such that 

OtiPkiKi) = Kl l<i<m. 
Now (7.10) follows from the relations 

0^=^Pu{f)=e{-no + Y.x{OLiPkm)^ 

and 

f = e {-no + ^ x{K^ , 

which in their turn follow from Definition 4.4 and Definition 3.2. □ 

The following lemma shows that, for functions u G L^(R"') such that u ^ u*, one 
polarization is enough to find a better approximation of n*. Its proof, left to the 
reader, follows the ideas of [7, p. 252] and repeats the proof of Lemma 6.4 in [14]. 
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Lemma 7.3. Let u G L^_^{W^), 1 < p < oo. If u = u^^ and u ^ for some G M 
such that ti < t, then there is a polarizer Hg = {y > s} with ti < s < t such that 

and 

\\UH^ — V^Wp < \\u — 

Lemma 7.4. Let u G L^(M"), 1 < p < oo. Then for every t G M there is a sequence 
of polarizations with polarizers {y > y^}, where yj. < t for all k = 1,2,..., such 
that the sequence of functions Um '■= OiLiPiU satisfies the minimality condition 

(7.11) - u*||p = min{||('u^)ii- - ■u*||p}, m = 1,2, . . . , uq = u, 

where the minimum is taken over all polarizations with polarizers H = {y > s} such 
that — oo < s < t. 

In addition, 

Um^u^ inLP{W). 

Proof. It follows from Lemma 5.2 [14] that the minimum in (7.11) is attained for some 
polarizer {y > yfc+i}, — oo < y^+i < t. Then by Lemma 6.1 [14], there are some 
function v G L^(]R") and a subsequence u^' such that 

Um' V in U'^W'). 

Then using the non-expansivity Lemma 3.1 wc obtain = u*. 

Now assume that v ^ u^. By Lemma 7.3, we can choose a polarizer H such that 

\\vh — U*\\p < \\v — ■u*||p. 

It follows that 

(7.12) \\{Um')H - U^Wp - \\Um' " U^\\p < \\iUm')H " VnWp + \\vh " U^\\p 

+ ||u^/ — v\\p — \\v — U*||p < 2||'U^/ - f lip + \\VH - U^Wp — \\v — u'^Wp 

—^\\vh — u^Wp — \\v — U*||p < 

as m' oo. 

On the other hand the sequence ||u^ — u*||p is monotonically decreasing. Hence, 

ll^^m+i ~ u^Wp — \\um — u^Wp ^ as m — >■ oo. 
Together with (7.12) this contradicts the minimality property (7.11). □ 

Lemma 7.5. Let u G C(M")nL^(M") and let Um be defined as in Lemma 7.4, whereby 
the condition (7.11) is satisfied with p = 1. Then 

Um^u* mC(M"). 

Proof. By Lemma 7.4, Um — 5" in L^^W"), and the functions Um are equicontinuous 
in view of Lemma 5.1 [14]. Because of Lemma 6.2 [14] we also have 

Urn' in CiW") 

for a subsequence Um' and Uy < u)u- Thus v = and the assertion follows. □ 
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8. RESCALING and LIMIT CASES 

The r*-transformation of sets and functions defined in Section 4 depends on the height 
t of the moving plane of symmetrization {y = t}, the range of which is — oo < t < oo. 
We recall that in its original setting as described in the Introduction the problem on the 
continuous symmetrization requires the standard range < i < 1 with the "boundary 
conditions" : 

(8.1) T^{n) = n, T^{n) = n*, 

where O* denotes (1, n)-Steiner symmetrization of O with respect to a fixed plane 
{y = to}- If to = 1) then the r*-transformation perfectly matches these conditions 
when restricted to the family 7Wn[0, 1], where 

Mn[a, b]:={neMn-- n(a, b)} 

and 

n(a, b) := {{x, y) G : a<y< b}, -oo < a < 6 < oo. 

IfQ £ A^„[a, b] with — oo < a < b < oo, the linear change of variables r = {t — a)/{b—a) 
gives the desired range < r < 1 with T^(yL) = and T^(ri) = f]*, where T'^ = T* and 
the asterisk denotes the (1, n)-Steiner symmetrization with respect to {y = b}. The 
case Q G A^„[— oo,6] with 6 < oo can be handled in a similar way with a non-linear 
change of variables, for instance, r = (1 + 5 — t)~^. 

The remaining case O G A^„[a, oo] with — oo < a < oo requires some additional 
consideration since the limit set 0,°° is not defined yet. One possible way to "correct" 
this defect is to use a post-composition T* = Sh^ o T* of the T*-transformation with 
a continuous shift Sh^ of (t — to)^ units in the direction of the negative y-axis. Here 
{t — to)'^ = max{0, t — to}. In this case, the limit set 0,°° = r°°(0) may be identified 
with the Steiner symmetrization ft* of Q with respect to the plane {y = to}. Of 
course, T* still possesses all basic properties of the T*-transformation. In particular, 
Lemmas 4.4 and 5.1 are still valid for T* for all — oo < t < +00. The case t = 00 
is included here because T°° is just the Steiner symmetrization with respect to {y = 
to}. Changing the variables via r = (I/tt) arctant + 1/2 we will have a family of 
transformations T"^ defined on the standard interval I = {t : < r < 1}. As we 
have seen in previous sections, the T*-transformations is continuous from the left for 
—00 < i < 00. Therefore the T*-transformation is continuous from the left on the open 
interval < r < 1. By this reason, the T'*-transformation will be called the continuous 
symmetrization defined on the standard interval. 

Being restricted to the class of bounded sets and functions with bounded supports, the 
T*-transformation is continuous at r = 1 as well. For unbounded sets and functions 
with unbounded supports the situation is different: some functional of domains or 
functions are continuous at r = 1 while some other such functionals are not continuous 
at r = 1. For instance. Lemma 5.1 is obviously not valid if Q is an unbounded set 
while the relevant Steiner symmetrization of is bounded. 

To define a one-dimensional T*-transformation for an arbitrary half-space H = H(a, n), 
we may combine the T*-transformation of Definition 4.3 with appropriate affine trans- 
formations of M". To be more precise, let A; > and let A be an orthogonal n x n 
matrix such that the operator L := k{A — Aa) maps H{a,n) onto {y > 0}. Then 
Tjj := o r* o L is the desired continuous symmetrization with respect to H. If 
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M G M-n is bounded then the scahng factor k can be chosen such that L{M) C -B^") 
and therefore when working with bounded sets M we may assume without loss of gen- 
erahty that M C B'^^\ Similarly if n G 5„ has a bounded support we often assume 
without loss of generality that suppu C B^'^\ 

We stress once more that any Tjj -transformation can be rescaled to get a Tjj -transformation 
defined on the standard interval I and that all basic properties of continuous sym- 
metrization discussed in Sections ^-7 still remain true for any T^^ -transformation. 



9. Continuous (A;, n)-STEiNER symmetrizations 

To define a one-parameter family of rearrangements Tj, left continuous in t G [0, 1] 
transforming sets and functions into their {k, n)-Steiner symmetrization for any 2 < 
< n, we suggest the following inductive algorithm: 

Let 2 < k < n. Suppose that for any (n — A; + l)-dimensional plane L' we have defined 
a continuous family of rearrangements transforming sets, functions, etc. into their 
(A; — l,n)-Steiner symmetrizations with respect to L' . Then let 5 be a (A, n)-Steiner 
symmetrization with respect to an (n— fc)-dimensional plane L. By Lemma 3.1, we can 
choose two intersecting (n — k + l)-dimensional planes Li and L2 defining (k — 1, n)- 
Steiner symmetrizations Si and S2 such that 

(9.1) S = lim (52 o Sif = lim Si o {S2 o Sif , 

where (^2 o Si)^ is the identity transformation. 

By our inductive assumption there are continuous (k — l,n)-Stciner symmetrizations 
T^'* and T^'*, t G [0, 1] into Si and S2, respectively. Let t{j) = {j — l)/j and let Ij = 

{t : t{j) < t < t{j + 1)}, j = 1, 2, . . . Then (ufL^Ij^ U {1} is a disjoint decomposition 

of the standard interval /. Next, for every j = 1,2,... and every t G Ij we define the 
transformation by 

(9 2) __ / T''- ° (^2 o Sif-' with r = if j = 2i- 1, 



[T'^^oSio{S2oSiy-' with r = jpi^^ ifj=2z, 
where i = 1, 2, . . . 



Inductive algorithm (9.2) defines a continuous transformation into a {k, n)-Steiner 
symmetrization S as follows. First, we choose a complete binary tree T of planes 
Loii...ii, 1 < Z < A; — 1, G {1, 2}, rooted at L = Lq such that for every 1 < Z < A; — 2 
and every multi-index ii . . . ii, the planes -Lon...i;i ^-nd ivoji...j;2 are (n — k -\- I -\- 1)- 
dimensional and the corresponding {k — l — l,n)-Steiner symmetrizations S'oii...i(i and 
Soi^...ii2 approximate Soij^,,,^ as in (9.1). 

Definition 9.1. By SC {k,n)-Steiner symmetrization corresponding to a tree T we 
mean a transformation Tj^ of sets and functions defined inductively by (9.2), where the 
initial 2^~^ continuous 1-dimensional symmetrizations are selected to be SC 1- sym- 
metrizations with respect to the corresponding hyperplanes Lqi^ ik-n where ii . . . ik-i G 
{l,2}'=-i. 
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Three remarks are in order now: 

Remark 9.1. The Sarvas approximation scheme used in the algorithm (9.2) may be 

replaced by any other approximation scheme. For example, one may use a sequence 
Si, i = 1,2, .. . of (1, n)- Steiner symmetrizations to approximate a given (k, n)-Steiner 
symmetrization: S = lim„_^oo 0?=i'S'i- In this case a r*-transformation can be defined 
as T* = o iOUSi) with T = {t- t{i))/{t(i + 1) - t{i)). 

Remark 9.2. One can define a continuous {k, n)-symmetrization by replacing Solynin's 
one dimensional symmetrization with Brock's one dimensional symmetrization. Al- 
ternatively, one may want to define a continuous {k, n)-symmetrization by "mixing" 
in the different stages of the algorithm (9.2) the SC symmetrizations with BC sym- 
metrizations (or with other types of continuous (1, n)-symmetrization if those will be 
discovered) . 

Remark 9.3. Any continuous deformation defined by the algorithm (9.2) is gener- 
ated by two kinds of transformations, continuous one dimensional symmetrization and 
[k, n)-Steiner symmetrization. Such a "hybrid" will inherit the properties that both 
its "parents" have and often the proof of a particular property of the transformation 
T^, for any fixed t, can be given via what we call the "standard inductive argument". 
More precisely the latter means that if a certain statement is known to be true for 
any {k, n)-Stcincr symmetrization, as well as for a corresponding continuous one di- 
mensional symmetrization, and if the corresponding property is invariant under trans- 
lations and scaling, then the corresponding result will remain true for any continuous 
fc-dimensional symmetrization defined by the algorithm (9.2). We will demonstrate 
how this "standard inductive argument" works in the proof of Theorem 9.1 below. 
We want to emphasize here that if the assumptions of some theorem or lemma deal 
with the varying parameter t, then the standard inductive argument is not valid 
and the proof of such statement is, usually, not so straightforward, cf. the proof of 
Theorem 9.2. 

Theorem 9.1. Let he an SC {k,n)-Stemer symmetrization into a {k,n)- Steiner 
symmetrization corresponding to the decomposition M" = M"^^ x and let ^, be 

a restriction of onto the slice M(xq) := {x = {x'Q,y) : y G M'^}. Then T^, respec- 
tively T^^i , acts on M", respectively on W{xq), as an open, compact, and smoothing 
rearrangement, which is continuous from the inside and from the outside. 

Proof. The result is already known for the one dimensional symmetrization, see 
Lemma 4.4. 

Assume now that the theorem is true for any integer n > 3 and some k — 1 such that 
2 < k < n. Now for a fixed < t < 1, let be a continuous symmetrization as 
described in the formulation of the theorem. If t = 1 or t = t{j) for some j > 2, then 
the T^-transformation coincides with the [k, n)-Steiner symmetrization or it coincides 
with a finite composition of {k — l,n)-Steiner symmetrizations, respectively. In both 
cases the theorem is known to be true. 

Let t{j) < t < t{j + 1) foT some integer j >2. Then according to the algorithm 9.2, the 
r^-transformation can be represented as a finite composition of {k — 1, n)-Steiner sym- 
metrizations postcomposed by a continuous (k — l,n)-Steiner symmetrization. Then 
using the known result for (k — l,n)-Steiner symmetrizations and the inductive as- 
sumption on the continuous {k — l,n)-Steiner symmetrizations we conclude that the 
theorem is true for any k, 1 < k < n. □ 
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The following lemma shows that continuous symmetrizations are monotone in measure 
on slices. Its proof easily follows from Theorem 9.1 and the non-expansivity property 
of Lemma 3.1. 

Lemma 9.1. Let Tj^ be a continuous symmetrization as in Theorem 9.1 and let J7 G 

U J". // < ti < t2 < ts < 1, then 

(9.3) d'{9^^{x')AVl^^{x')) < jC''{n^'{x')An^^{x')) 
for every x' G R"~'^ and therefore 

(9.4) {n^^AQ*^ ) < (J^*i AJ^*3 ) . 

Theorem 9.2. Let be a continuous symmetrization as in Theorem 9.1 and let tg, 
s = 1,2, . . . be an increasing sequence such that tg G [0, 1] and ts to. Then 

(9.5) d{dn^^ , dn^") ^0 as s^oo 
if ft is a bounded open set and 

(9.6) d(Jl*%0*o)^0 as s^oo 
if ft is a compact set. 

Proof. To be specific, we will assume that is a bounded open set. In the case of 
compact sets the proof is easier and is left to the reader. 

We proceed by induction. For k = 1, the statement is true by Lemma 5.1 modulo 
some remarks in Section 8. Suppose that for every SC (j, n)-Steiner symmetrization 
with 1 < j < A; — 1 the conclusion of the theorem is true. Our goal now is to prove 
that the theorem is true for j = k. 

If to < 1, then t{m) < to < t{m + 1) for some positive integer m. By Definition 9.1 
and Algorithm (9.2), the set is the image of under some {k — 1, n)- 

Steiner symmetrization 5*1 and moreover there exists a continuous {k — l,n)-Steiner 
symmetrization T^''^ into Si such that 

(9.7) 17* = ri'^(17*M) for all t{m) < t < t{m + I) , 

where r = T{t) = {t — t{m))/{t{m + 1) — t{m)). Since T{ts) T{to) as s ^ oo, the 
theorem follows from (9.7) and the inductive assumption. 

In the case to = 1 the proof is by contradiction. If (9.5) does not hold, then as in 
the proof of Lemma 5.1 we can find e > and, if necessary, subsequences (for which 
we keep previous notation), tg and ftg = ft*", and a sequence of points Xg G M" with 
— )■ xo G M" such that one of the following two conditions is satisfied: 

(a) Xg G dfl* and dist(2:s, dftg) > e for all s = 0, 1, 2, . . . 

(b) Xg G dflg for s = 1, 2, . . . and dist(xs, dft*) > e for all s = 0, 1, 2, . . . 

In the case (a) we consider two subcases: 

(i) Si"^ (xo) n = for all s = 1, 2, . . . 

(ii) Si"^ (xo) C ft*" for all s = 1, 2, . . . 
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Case (i). Let t(ms) < tg < t{ms + !)• In the case under consideration there is (5o > 
such that 

(9.8) {^t^AVt*) > So 
for ah s = 1, 2, . . . Since 

by equation 3.11 of Theorem 3.1 we have 

(9.9) c (^r2*(™')Ar2*("^+i)) < c"- (|o*(™=)aq*) +£"- (^0*(™-+i)Ar2*) ^ 

as m — )• oo. Next by Lemma 9.1, 

(n^- An*^'^'+'^'^^ < CP- (^j^*('"'')aj)*('"''+^)) , 

which together with (9.9) contradicts (9.8). 

Case (ii). Let xq = (xq,?/o), where G W^~^. Let ^*{x'q) and ^{x'^ denote the 
restrictions of ^7* and Jl* respectively in the shce M^./ , each of which is not empty by 
the condition (ii). Since 0*(xo) is a (/c, A;)-Steiner symmetrization of ^{x'^^ it is a 
nonempty A;-dimensional ball. By assumption (a), G dVi* and therefore 

/:'=(S«(yo)\J^*(4))>5i 

for some b\ > 0. Hence, 

(9.10) £''{n*{x'Q)An^^{x'Q)) >6i, s = l,2,... 
By inequality (9.3) of Lemma 9.1 we have 

c''{n*'{x'o)An'^"''+^\x'o)) < jC''{n*^'^'\x'o)An*^"''+^\x'o)) . 

By equation 3.11 of Theorem 3.1, 

C''{n^^'"'\x'f^)An*{x'f^)) ^0 as m ^ oo. 

As in the case (i), the latter two relations contradict (9.10). This completes the proof 
in the case (a). 

Now we turn to the condition (b). In this case equation 3.10 of Theorem 3.1 implies 
that there is a neighborhood U of xq such that one of the following two conditions is 
satisfied: 

(i) Unn* = $ and U n = for all m large enough, 

(ii) U cO,* and U C for all m large enough. 

For S > 0, T > 0, and xq = (xq, yo) with yo £ let 

Qsixo) = Bf-'^Hx',) X M^ Q(xo,5,r) = i?f-')(x^) x 4^). 

Considering case (i), we first assume that |yo| > 0. Since for every x' G W^~^ the 
section Q.*{x') is a ball in M'^ centered at the origin, condition (i) implies that we can 
find (5o > and tq > sufficiently small such that 

(9.11) ri* n (35(,(xo) C (5(xo,(5o, |yo| - To) and n Q5o(xo) C (5(xo, (5o, |yo| - tq) 

for all m large enough. According to (9.2) the set 47*" is obtained as a result of the 
action of a certain continuous (A; — l,n)-Steiner symmetrization of the set ri*("^'=) into 
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the set Since the intersections n*^"^*) n Qsoixo) and n Qsoixo) 

both belong to Q{xo,do,\yo\ — tq) and since the set Q(a;o, 5o, |yo| — tq) is invariant 
under the continuous {k — l,n)-Steiner symmetrization mentioned above, it follows 
from the property of monotonicity in slices of the continuous symmetrization that 
ri*" n Q5o(xo) C (5(xo,5o) |yo| — for all s sufficiently large. The latter contradicts 
the assumptions Xs G 5^2*" and ^ xq. 

If ?/o = then, in the case (i), VL* ^Q^^^{xq) = for some Sq > 0. Hence the intersection 
Q n Qso{xo) is also empty and therefore O*" n Qsgixo) = for all s, which again 
contradicts the assumptions Xg G dQ^^ and Xg — )• xq. This finishes the proof in the 
case (i). 

In the case (ii), we argue as above to conclude that there are Sq > and tq > such 

that 

(9.12) Q(xo,5o, jyol + To) C ri* n (55„(a:o) and Qixo,5o,\yo\ + tq) C Qmf^Qsoixo) 

for all m sufficiently large. As above, the monotonicity property of the continuous 
(A; — l,n)-Steiner symmetrization from ri*^"*'') into U'ti'^^+'^) shows that Q{xo,5o,to) C 
$7*" for all s large enough. The latter contradicts the assumptions Xg G dU^^ and 
Xs — >■ xq. The proof of the theorem is complete. □ 

Lemma 9.2. Let J7,r2' G Qn,b be such that C O and let (•)* denote an SC sym- 
metrization into the {k,n)-Steiner symmetrization Sym corresponding to the decompo- 
sition M" = M"-^ xR'^. Then for every t, there exists a finite number of transformations 
Pi, ... , Pn such that 

n'j^ := Pi o . ■ ■ o p,^ {n') c 

and 

dist(50',9f^) < dist(9Q^,5f^*), 
where each Pj is either a polarization with the polarizer Hj such that := dHj D M""'^ 
or a shift in a direction I such that I _L M"^'^. 

Proof. If fc = 1, then the considered symmetrization is one dimensional and the lemma 
follows from Corollary 7.1. 

Let k > 2. If t = 1 or t = t{j) for some j > 2, then the T^-transformation coincides 
with the (k, n)-Stcincr symmetrization or with a finite composition of (k— 1, n)-Steiner 
symmetrizations. In either case the result is known to be true, cf. Lemmas 7.1 and 
7.2 [14]. Therefore for any fixed t, the desired conclusion follows via the standard 
indTictivc argument as we explained in Remark 9.3 and demonstrated in the proof of 
Theorem 9.1. □ 

10. Integral inequalities 

Many integral inequalities well-known in the theory of symmetrization remain valid 
for the continuous symmetrization as well. We start with the following Z^-continuity 
lemma. 

Lemma 10.1. Let u G Lq^(M") , 1 < _p < oo and let = T^{u), < t < 1 denote a 
continuous {k,n)-Steiner symmetrization. Then the mapping t ^ is continuous in 
LP -norm; i.e., 

(10.1) lim||u^+*-u"||p = for all < s < 1. 
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Proof. First we assume that is a step function. Then 



with some 5 > Q and some measurable sets Ei D ... D E^. Let EJ = T'^{Ei), 
< r < 1. By Lemma 9.1, we conclude that 



i=l 



In general, u can be approximated in the I^-norm by step functions Um- Then using 
the inequality 

||«^+* -u^< - + - <|| + IK - «^|| 

and the non-expansivity Lemma 3.1 we obtain (10.1). □ 

Our next lemma shows that a similar continuity property remains valid for the space 
of continuous functions. 

Lemma 10.2. Letue Co(M")nL^(M''). Then for every 0< s<l, ||u*+*-u^||oo ^ 
ast^O. 

Proof. Fix < s < 1 and e > 0. Since u G Co{W) n L\{W), we can find v G 
Wq'°°{R"-) n L^(M") such that \\v — u\\oo < e/3. Then using the non-expansivity 
property of Lemma 3.1 we obtain for all t small enough 

\\u'-u'+*\\oo < ||«'-'y^||oo + ||'y'-'y^+*||oo + ||'y'+*-'"^+*||oo 

< 2||u-i;||oo + ||'y^-^^*"^*||oo <£, 
which proves the lemma. □ 

The rest of this section contains seven theorems. The first six of them are monotonicity 
statements that correspond to the inequalities, which are well-known in the theory of 
Steiner symmetrization. Their proofs in all cases follow the same scheme. First we use 
an approximation by polarizations to prove the required monotonicity for the contin- 
uous SC 1-symmetrization. Then we apply the standard inductive argument to show 
that the same kind of monotonicity holds for the continuous (/c, n)-symmetrization for 
any k. 

Theorem 10.1 below is related to the well-known convolution type inequalities, cf. [14, 
Lemmas 8.1, 8.2], [8], [6, Corollary 2]. It shows that convolutions are, in fact, monotone 
functions of the parameter of the corresponding continuous symmetrization. 

Theorem 10.1. Let u,v,w G 5+ with w = w* , where ( )* denotes a {k,n)-Steiner 
symmetrization and let j be a Young-function. Then the integral 



II 



j(\u\x) - v^{y)\)w{x - y) dxdy 



decreases in < t < 1 provided that the integral converges for t = 0. 
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Proof. First we prove the theorem for the SC 1-symmetrization. By the semi-group 
property (6.7), we have to prove the inequaUty 

(10.2) / / j{\u^(x) — v^{x)\)w{x — y) dxdy < / / j{\u{x) — v{x)\)w{x — y) dxdy 



for — cx) < t < oo. 

First observe that in view of non-expansivity Lemma 3.1, we can restrict ourselves to 
the case that u and v are continuous functions with bounded support. Then we define 
inductively two sequences V'ln and of polarizations of ii and v as in Lemma 7.4, 
where the corresponding half-spaces are chosen in such a way that the minimality 
property (7.11) is satisfied. By Lemma 7.5, the sequences Um and Vm converge in 
C(]R"') to and u*, respectively. Then (10.2) follows by applying Lemma 8.1 in [14] 
inductively. 

To get values of t varying in the standard range < t < 1, we may use scaling and 
translation as it is explained in Section 8. Of course, the latter two operations do not 
change the integrals in (10.2). 

Finally, it is well known that the desired monotonicity result holds for the {k, n)-Steiner 
symmetrization for any k, I < k < n, see [14, Lemma 8.2]. Therefore for k > 2, the 
proof of Theorem 10.1 follows via our standard inductive argument. □ 

The Dirichlet-type inequalities for functions and their (/c, n)-Steiner symmetrizations 
also admit continuous counterparts. 

Theorem 10.2. Let u G W+'^(M") nS+, l<p< +oo. Then G wl'^(m") D S+ for 
all <t < 1 and ||V'u*||p decreases in < t < 1. 

Furthermore, if V is some linear subspace which either contains all "y- directions" 
Xn-k+ii - ■ ■ 1 Xn, or is orthogonol to each of these directions, then ||Vyu*||p decreases 
m < t < 1. 

Proof. For the (fc, n)-Steiner symmetrization this result is well known, see [14, The- 
orem 8.2]. So, we give the proof for the case of the SC 1-symmetrization. Then for 
A; > 2, Theorem 10.2 will follow via the standard inductive argument. 

Let denote the SC 1-symmetrization of u. The semigroup property (6.7) shows that 
it is enough to prove the inequalities 

(10.3) ||Vu*||p < ||V||p and ||Vyu*||p < ||Vyu||p 

for — oo < t < oo. 

For a fixed t, let Um be the sequence of polarizations of u defined by Lemma 7.4, which 
converges to tt* in L*'(M"). We consider two cases. 

(i) Let 1 < p < oo. Since ||VM,n||p = ||Vm||p, by Lemma 5.3 [14] we can find a function 
V G W^'P{M."') and a subsequence Um' such that 

Um' V weakly in W^'P{W). 

This means that for every tp G Co°(M") and i = 1, . . . ,n, 

I ipvxidx^ I V3^^^dx = - /" ifxiUjn' dx ^ - f ip,j;V dx, 

that is, V = u*. In view of the lower semi-continuity of the norm it follows that 

||Vu*||p < liminf ||V('Urn)||p = llVuHp 
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that is the first inequahty in (10.3). Using equation (5.10) of Lemma 5.3 [14] one can 
prove the second inequality in (10.3) analogously. 

(ii) Let p = 1. By Lemma 5.3 [14], the functions [V-Urnl and [V^l are rearrangements 
of each other. This means that for every 6 > 0, 

sup I J \{um)xi\dx : ^{E) <5^< sup!^J \Vum\ dx : ^{E) < 6 

= supjy \Vv\dx: L"(£;)<(5|. 
Hence, if E^. is any sequence of measurable sets with lim(L"(£^fc)) = 0, we infer that 
sup|y \{um)xi\dx: m G n| — )■ ask 



oo. 



Applying a well-known weak compactness criterion in L^(M"), see [3, p. 199], we again 
can extract a subsequence Um' converging weakly in W^'^(W^). Finally, proceeding as 
in case (i) the assertion follows in the case p = 1 too. □ 

The following theorem gives an analog of the previous theorem for the spaces of con- 
tinuous functions. 

Theorem 10.3. Let u G C(]R") fl 5+. Then u,,t decreases in < t < 1. 



Proof. Let u G L5j_(M"'). For the SC 1-symmetrization the result follows from the proof 
of Lemma 7.5. In the general case we choose a sequence of functions in C(M") H 
L^(]R") converging to u in C(M"). Then applying the non-expansivity Lemma 3.1 we 
obtain 

ll(u^)* - u*||oo < ||u^-ii||oo, m = 1,2,... 
and the assertion for the SC 1-symmetrization follows. For A; > 2, we apply the 
standard inductive argument. □ 

It is also easy to prove the monotonicity property of convex functionals. 

Theorem 10.4. Let u G W_]:'"^(M"^) and let j he a Young-function such that 

/[gn id Vii|) < oo. Then the integral mean Jjg„ j(| Vu*|) (ia; decreases m < t < 1. 

Furthermore, ifV is a linear subspace which either contains all "y- directions" Xji—k+i^ ■ ■ ■ ) •^m 
or is orthogonal to each of these directions, then Jt^„ j(|Vyw*|) dx decreases inO <t < 
1. 

Proof. The standard inductive argument still works. Therefore, it is remains to prove 
the theorem for the SC 1-symmetrization. 

Fix t, — oo < t < oo. By the semigroup property (6.7), we have to prove the inequalities 
(10.4) / j{\Vu'\)dx< [ ji\Vu\)dx, [ j{\Vvu'\)dx< [ j{\Vvu\)dx. 

JR" 

Assume first that u G Co:^^(M"). If we choose a sequence of polarizations of u converging 
to in L^(R"), we conclude from Lemma 5.3 [14] that 

/ ji\Vum\)dx= [ ji\Vu\)dx. 



/ 
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Because of the weak lower semi-continuity of the integral functional this leads to 

/ j{\Vu\) dx = limmf j{\Vum\)dx> j{\Vu''\)dx. 

Uue VF+'^R"), we choose a sequence Vm G Cq'^CR"^) such that 

Vm^u in W^''^{W) 

and 

j{\Vvm\)dx-^ I j{\Vu\)dx. 

This means that we have for a subsequence v^' 

{vm'Y weakly in W^'^{MP), 

and we conclude by the weak lower semi-continuity of the functional that the first 
inequality in (10.4) holds true. 

One can prove the second inequality in (10.4) analogously. □ 
An analog of Theorem 10.2 also holds for SF-functions. 

Theorem 10.5. Ifu G BV{MP)C^L\{W), thenu* G BV{W)C^L\{MP) and \\Du*\\bv 
decreases m < t < 1. 

Proof. As in the previous theorems, we have to prove the result for the SC 1- 
symmetrization. By the semigroup property (6.7) we have to prove that 

(10.5) \\Du*\\bv < \\Du\\bv 

for any fixed t, — oo < t < oo. 

We choose a sequence of functions u„i G W^'^(M") which converges to u in BV{W^). 
By Lemma 5.3 [14], the functions [umY arc equibounded in W^'^{M.'^). Therefore there 
are some function v G BV{M.)"' and a sequence Um' such that 

(tt^/)* ^ V weakly in By(M"). 

On the other hand from the inequalities 

WiUmY - U^Wl < \\Um - u\\i 

we conclude that 

{umY in Li(R"). 

Now let Hi denote the Radon-measure which is associated with the weak partial deriv- 
ative Vxi, i = 1, ■ ■ ■ ,n. Then we have for every ip G Cq°(R"'), 

I ipdi^i^ [ ^ ^(^^"•^ ) ) ^ _ f ^ _ f ip^,u^dx, 

which means that v = u*. 

Finally, the weak lower semi-continuity of the norm gives 

\\Du^\\bv < liminf ||V((C/^)*)||i = lim || Vtx^||i = \\Du\\bv- 

□ 

Choosing for u in Theorem 10.5 the characteristic function of a set of finite perimeter 
we derive the following "monotonic isoperimetric inequality" in M". 
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Theorem 10.6. Let E is a Caccioppoli-set in M". Then the perimeter \\Dxe*-\\bv of 
£■* is a decreasing function in < t < 1. 

Now we prove that the mapping t i-> «* is continuous from the left in Sobolev-spaces 
VF^'^(R"'). It is worth mentioning here that an analog of this result does not hold in 
the space BV{W^) H L^{W^). The characteristic function u = of a single interval 
/ C M provides a simple counterexample in dimension n = 1. 

Theorem 10.7. Let u £ Wl'^{W'), 1 < p < oo and let tm, m = 1,2,... be an 

increasing sequence in (0, 1) such that t„i t a.s m — > oo. Then 

(10.6) u*"" in VF^'f (M"). 

Proof. First we consider the case p > 1. By Lemma 10.1, we have u*"* — >■ u* in L^iW^). 
From this we conclude that 

(10.7) M*"' n* weakly in M/^'P(M"). 

In addition, Theorem 10.2 combined with the weak lower semi-continuity of the L^- 
norm implies 

(10.8) lim ||Vu*'"||p = II Vu* 



Since for p > 1 the spaces W^'P{M."') are uniformly convex, (10.6) follows from (10.7) 
and (10.8). 

Now let p = 1. We note that (10.7) and (10.8) remain true for p = 1. Then we fix 
an index i G {!,... ,n} and set Vm '■= {u*"')xi: v := {u^)xi- Let j{z) := VT+^ — 1. 
Then j{z) is a Young-function which satisfies the inequality j{z) < z. Therefore 
Theorem 10.4 and the weak lower semi-continuity property of the integral imply that 

(10.9) lim / {^/l + v^-l)dx = [ {VlTv^-l)dx. 

Applying Taylor's formula with a reminder we obtain 

/ (a/1 + < -l)dx> / {y/lTv^ - 1) dx + ^ (vm - v) dx 

2U.{l + clfl^ 

where = maxji;^, i;^}. Passing to the hmit as m — )■ oo we obtain 



hm / dx 

m^oo Ln (1 + c2j)3/2 



The latter equation implies that for every positive integer fc, 

(10.10) lim / \vm -v\dx = 0, 

where F^^ra = {a; G M" : \vm\ < \v\ < k}. Since Vm ^ v weakly in ^^(R") we find 
that 

(10.11) lim / \vm\dx = 



uniformly over all m. Here Gk,m = {a; G M" : \vjn\ > k}. 
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Now (10.6) follows from (10.9) an (10.11) via the inequality 

||^^m~'y||l^ / \Vm—v\dx + 2 / \Vm\dx + 2 / |u| , 

which holds for all positive integers m and k. □ 

11. Comparison theorems 

First we introduce a partial order ^* related to the continuous (fc, n)-Steiner sym- 
metrization T*. 

Definition 11.1. For functions f,g& L^[W^) and < t < 1, we write 

(11.1) f g if and only if f fhdx < [ g^h^dx for all h e {W) . 

Jm" Jr" 

The following theorem generalizes the well known equivalences in the theory of sym- 
metrization (see [14, Remark 10.1], [4]) to the case of SC 1-symmetrization. 

Theorem 11.1. Let f,g G L^(R") and let (•)* denote the SC 1-symmetrization, — oo < 
t < oo. Then the following relations are equivalent: 

(11-2) f^'g 

(11.3) / f*h*dx< ! g*h*dx for all h e L"^ {W) , 

(11.4) j{f) -<* j{g) for all Lip schitz- continuous Young-functions j, 

ry ry 

(11.5) / f\x', s)ds< / g\x', s) ds for all x' G M""^ and every y>t, 

J2t-y J2t-~y 

(11.6) f\x', y) < g\x', y) for all x' G M""^ and every y>t. 

Proof, (a) To prove that (11.2) implies (11.5), we fix c > and set Mc := {/ > c}. 
Then for fixed x'q G M"~^ and positive e, we define a function 

he{x',y) := K{x',y)ipe{x',y) 

with X = {x', y) G and 

= Ve{x',y) := X{{{x',y) ■ \x' - x'ol < £», 

1 if f{x', y)> candy < y{x', t) 
otherwise, 

where the separating function y{xQ,t) = yMc{x'o,t) is defined in Section 4. 
Now (11.2) implies 

ryix',t) 



K{x',y) :-- 



r ry{x ^t) r r 

/ / f\x',s)dsdx' = / f\hefdx= / fh^dxK 

r r fy{x',t) 

/ g\heYdx= / / g\x',s)dsdx'. 

Jr" Ji\x'-xL\<e} J2t-y(x',t) 



'{\x'-x'o\<£} J2t-y{x',t) 

Taking the limit here as e — )■ O"*", we obtain 

ry{x'o,t) ry{x'o,t) 
/ f\x'Q,s) ds< / g\x'Q,s) ds, 

J2t-y(x'n,t) J2t-y(xL,t) 
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where y{xQ,t) = yMci^ojt) depends on c. Since c > can be chosen arbitrary small 
the latter inequality implies (11.5). 

(b) To show that (11.2) imphes (11.6) we fix (xQ. yo) G with yo > t. Then using 
notation of part (a) of this proof we note that the separating function y{xQ,t) = 
yMci^'ojt) depends monotonously on the height of polarization c. Therefore we can 
choose c > small enough such that yo > y{xQ,t). Then we have f^{xQ,y) = f{xQ,y) 
for all y > yo- Now wc choose the function h in (11.1) to be the Dirac ^-function at 
xo = {x'Q,yo), i.e. h := 6{xo). Then we have h = h^. Finally applying (11.1) we derive 

f\x'o,yo) = f{x'o,yo) = / fhdx< / 5*^* = 5*(4,yo), 
which is (11.6). 

(c) Now we show that (11.5) and (11.6) together imply (11.3). First we assume that 
h = x{^) is a characteristic function of M G M.{'R"'). Let y{x',t) = yM{x',t) and let 
M{x',t) = M(x') n{s: s> yix',t)}. Then (11.5) and (11.6) imply 

f r { ry(^''*) r 1 

(11.7) / fdx= { f\x',s)ds+ f\x',s)ds}dx'< 

Jm* Jm"-i [J2t-y{x',t) JM{x',t) J 

ry(a^':*) 

l2t-y{x',i) JM{x',i) 

Next let ^ be a step function, i.e. 



f ( fy{^',t) f If 

/ < g\x',s)ds+ / g\x' , s) ds \ dx' = / . 

7m"-i J2t-y(x't) JM(x't) Jm* 



I) 



with some e > and some sets Mj G A^(M") such that Mi D • • • D M^. Applying 
inequality (11.7) to the functions x(Mj) we obtain the desired inequality: 



/ /Vdx = eV/ fdx<sy2 9^dx= dx . 

Finally, every /i G L^p(M") can be approximated by step functions. Therefore, in the 
general case (11.3) follows from the previous inequality. 

(d) To prove that (11.3) implies (11.4), we may assume without loss of generality that 

J G C^. For p G L5p(R"), we set h := j'{f)p. Since j' is an increasing function we have 
/i* = j'{P)p^- This equality combined with (11.3) implies 

(11.8) / fj'{f)p'dx< f g'j\f)p'dx. 

Since j is a convex function we have 

which together with (11.8) leads to (11.4). 

(e) Finally, (11.2) is a special case of (11.4). □ 

The following corollary shows that equivalencies (11.2) - (11.4) remain valid for con- 
tinuous {k, n)-Steiner symmetrization for all A;, 1 < A; < n. Its proof follows from 
Theorem 11.1 via the standard inductive argument. 
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Corollary 11.1. Let f,g E L^(]R") and let (•)*, where <t < 1, denote the contin- 
uous {k,n)-Steiner symmetrization. Then equivalencies (11-2) - (11-4) remain valid 
for allO<t<l. 

Corollary 11.2. Let f,g € and let (•)* denote the continuous {k,n)-Steiner 

symmetrization into a given {k,n)-Steiner symmetrization (•)*. Then for all s and t 
such that < s < t < 1, the following implication holds: 

If f g, then f ^* g. 

In particular, if f ^* g, then f -<* g. 

For the SC 1-symmetrization Corollary 11.2 follows immediately from the semigroup 
property (6.7). Then for any A; > 2, its proof follows via the standard inductive 
argument. 

Now wc prove two comparison lemmas concerning partial symmetry of solutions of 
certain elliptic and parabolic PDE's. These results and their proofs show, in fact, 
that the approach to comparison theorems in partially symmetric domains, based on 
the continuous symmetrization, is a closed relative of the Alexandrov's moving plane 
method, sec [26] and [18]. In the context of comparison theorems the approach based 
on a continuous symmetrization for the first time was used by Solynin [27]. 

Lemma 11.1. Let C M" he a hounded domain, c > 0, f,g & L^(r2), and let u,v he 
solutions to the following boundary value problems: 

(11.9) u,v eW^'^ip,), -Au + cu = f, -Av + cv = g in n. 

For < t < 1, let (•)* denote a continuous {k,n)-Steiner symmetrization. If for some 
0<t<l, n = n\ f = f \ g = g\ and f ~<* g, then 

(11.10) u = u\ v = v* 
and 

(11.11) u^^v. 

Proof. First, we prove the lemma for the SC 1-symmetrization. Then, of course, 
— DO < f < oo. Equality (11.10) follows easily from the maximum principle. 

To prove (11.11), we may assume without loss of generality that f,g& Cq!^{^). Then 
u and V are classical solutions which are smooth in (7. 

Let G = {x = {x', y) : {x', 2t — y) G ^1} and let w = u — v. For (x', y) & G and e > 0, 
we define a function 

Ws{x',y):= / [ w{x' , s) + w{x' ,2t — s) — sup vu — s\ ds. 

Jt \ dGnHt J 

Since f <*' g one can easily see that 

(11.12) -AWe + cWe<Q inG. 

In addition, wc have = on dHt. Since solutions of (11.12) satisfy the maximum 
principle we have: sup(^ = ^^VdGnHt Since = i7* one can easily see that the 
domain G is convex in the direction of y-axis. Therefore the positive direction of y-axis 
points outward the domain G on dG H Ht. Since < —e on dG H Ht we conclude 
that < in G for all £ > 0. Taking the limit as e — >■ 0+, we obtain 

(11.13) Wo < in G. 
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Let US show that w < in If not, then supf2_(j) w; > 0. Since —Aw + cw < 

in Cl-(t) and w = on D Hf the maximum principle implies that sup^^^j w = 
supQfj^ w. The second supremum here is attained at some point xq £ dHt with w(xq) > 
0. Since w{xo) > supQQp^jj^ w, the inequality w{xo) > implies that Wo{x) > for 
a; G G in a small neighborhood of xq, which contradicts (11.13). 

Therefore we have u; < in Now (11.11) follows from (11.5), (11.6), and (11.13). 

Now for A; > 2, the lemma follows via the standard inductive argument. □ 

Similar lemma holds also for parabolic problems. Its proof follows along the lines of 
the proof of Lemma 11.1. Therefore the details are left to the reader. 

Lemma 11.2. Let (•)* denote a continuous {k,n)-Steiner symmetrization, < s < 1. 
Let c > 0, T > and let he a bounded domain in M" such that = 17'^. Let functions 
uq,vq G L'^(^) and f,g G L^(r2x (0, T)) satisfy, respectively, the following conditions: 

Uo = Uq, Vq = Vq, Uq Vq 

and 

f{.,t) = f%;t), g = g{.,t)=g%.,t), f {■ ,t) g{. ,t) for all t G {0,T) . 

Further let u,v G L2(0, T; T4^o '^(Q)) n C{[0,T]; L^(n)) be solutions to the following 
initial boundary value problems: 

(11.14) ut- Au + cu = f, vt- Av + CV = g, in n x (0, T), 
u{x,0) = uo{x), v{x,0) = vo{x), in Q. 

Then 

(11.15) u{-,t) = u%;t), v{;t) =v%;t) for all t E {0,T) 
and 

(11.16) u{-,t) v{;t) for allte {0,T). 

To simplify some notations in our formulations and proofs, we will use the following 
definitions from [14]: 

Definition 11.2. Let J7 C M" 6e a bounded open set, c > 0, and f G L^(ri). We say 
that u solves the problem Bi(f2, c, /) ifu is the solution to the following boundary value 

problem: 

(11.17) uGWq'^{Q), -Au + cu = f inn. 

Definition 11.3. Let U C be a bounded open set, c > 0, / G ^+(f^) and 7 : — 
Wq be a continuous and nondecreasing function. We will say that u is a solution 
of problem B2(J^, c, 7, /), if u is the nonnegative minimal solution of the following 
boundary value problem: 

(11.18) ueWo'^{n), u>0, -Au + CM = 7(n) + / in Q, 
that is, 

(i) u is a solution of the problem (11.18), and 
(ii) {) <u<u for all other solutions u of (11.18). 

For a brief discussion of important properties of the solutions to the problems in 
Definitions 11.2 and 11.3, we refer to [14, Section 9]. 
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Theorem 11.2. Let (•)* denote a continuous {k, n)-Stemer symmetrization, <t <1. 
Let Q, C M" be a bounded open set, c > 0, / € L'^{Q), and let ^ be a Young function. 
For a fixed t, < t < 1, let g £ L^(ri*) be such that g^ = g and f -<* g. Let u and v 
be the solutions to the problems i?2(^,c, 7, /) and 52(ri*, c, 7, 5), respectively. Then 

(11.19) v = v^ 
and 

(11.20) u -<* V. 

Proof. As well known, the assertion of this theorem holds true for the {k, n)-Steiner 
symmetrization for any k, 1 < k < n; see, for example. Theorem 10.1 in [14]. 
Thus, in view of the standard inductive argument we have to prove it for the SC 
1-symmetrization only. Then, of course, —00 <t<oo. 

(1) First we assume that 7 = and that / is a simple function with compact support 
in Q., see Definition 7.1. For a fixed t G M, let v denote the solution to the problem 
Bi(r2*, c, /*). The maximum principle tells us that v = and v = v^. Furthermore, 
let h be an arbitrary function in L^(f2*) satisfying h = h^, and let w be the solution of 
the problem Bi(ri*, c, h). Since again w = w*, w > 0, and / ^* g, we find after partial 
integration that 

/ vh^ dx = wf^ dx < wg dx = vh^ dx, 
Jn* Jo* JQt jQt 

which means that 

(11.21) v^*v. 

Next let Vl' be an open set such that supp (/) C ri' C fi' C By Corollary 7.1 and 
Lemma 7.2, we can find a finite number of polarizations Pi with polarizers Hi = {y > 
yi}, i = l,...,N, where 

yi <y2 < ■■■ <yN <t, 

such that the closure of OfL^Pi{n') is in and (Oili^J)* = /*• 

Let r^TO = Q^iPiO,' and fm = OiLiPif: m = 1, . . . , N. Let u' and tt^ be the solutions 
to the problems Bi(0', c, /) and Bi(ri^, c, fm), respectively, m = 1,...,N. Applying 
Theorem 9.1 [14] we conclude that 

(11.22) u' -<Hi u[ -<H2 u'2 ~<H3 <Hn ^*Ar> 

where denotes the partial order related to the polarization with the polarizer 

H, see [14, p. 1783]. Since C ri* and /at = /*, the solutions ■u'jy and v satisfy the 
inequality 

< u'j^ < V a.e. in fijv 
Together with (11.21) and (11.22) this implies that u' ^* v. 

Now we choose open bounded sets n'' such that Q'' C n^^^, = 1, 2, . . ., and Ukfl'' = 
Let n* denote the solution of the problem Mi{il^,c, /), k = 1,2,.... By the above 
consideration we have v, k = 1,2, . . .. By the convergence property of elliptic 

boundary value problems in varying domains, see Lemma A [14], the sequence u'' 
converges to u in L^(0). This proves the assertion in the case under consideration. 

(2) Next we assume that 7 = but / is an arbitrary function in L'^{Q). Since simple 
functions are dense in Li(fi), there is a sequence of simple functions /*^, k = 1,2,..., 
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with compact supports in U, such that /'^ ^ / in L^(J7). Then we can find open sets 
n'', k = l,2,..., such that supp (f) C i^'^ C C 0^+^ C O. and U^Q^ = n. 

Let ■u'^ and Vk be the solutions to the problems Mi(p,'^ , c, f'^) and ]Bi((r2'^)*, c, (/'^)*), 
respectively, A; = 1,2, By part (1), we have 

(11.23) Uk-^^vk, A; = 1,2,... 

By Lemma 7.4, (Z'^)* — >■ /* in L^(n*) and also by the monotonicity property of rear- 
rangements, 

closure {{n'^Y) C (J^*^+^)* C and Ufc (0^=)* = J^*. 

Therefore by the convergence property of elliptic boundary value problems in varying 
domains, see Lemma A [14], we have 

Uk ^ u in L^(J7) and Vk ^ v in L^(f2*). 

This together with (11.23) proves the theorem in the case 7 = 0. 

(3) Next let 7^0. According to Remark 9.2 in [14] we approximate u and v by 
solutions to the problems ]Bi(r2, c, 7(^^01-1) + /) and Bi(r2*, c, "y{vm-i)+g), respectively, 
m = 1, 2, . . .. Here ^0 = ^0 = 0. Assume that we had proved that Um ^* Vm for some 
m. Notice that for m = this is trivial. Then by equation (11.4) of Theorem 11.1, we 
obtain that j{um) + / ^* 7(^m) + 9- By parts (1) and (2) of this proof this means 
that also Um+i ^* Vm+i, and we conclude by induction. □ 

Theorem 11.2 and equivalence relations of Theorem 11.1 lead to the following corollary. 

Corollary 11.3. Let u and v be solutions defined in Theorem 11.2. Then for every 
Young function j, 

(11.24) / j{u)dx< / jiv)dx, 

Jci Jn* 

if the above integrals converge. In particular, 

(11.25) ll^llp < ll^llp for all 1 < p < 00. 

One might ask under which conditions the equality holds in inequalities (11.24) and 

(11.25) , and believe that equality is possible only — roughly speaking — if possesses a 
partial symmetry. For Steiner symmetrizations this result was proved in [14] . To prove 
this uniqueness result for the continuous {k, n)-Steiner symmetrization, we restrict 
ourselves to the case where f2 is a domain. 

Theorem 11.3. Let (•)*, < t < 1, denote the continuous {k,n)-Steiner symmetriza- 
tion into some {k,n)-Steiner symmetrization with respect to a plane E. Let il, fi*, c, 
f, g, 7, u, and v be as in Theorem 11.2 and assume that is a bounded domain and 
/ > on Assume that there is some Lipschitz continuous Young function j which 
for some t, < t < 1, satisfies 

(11.26) f j(u)dx= f j{v)dx>0. 

Then = il* and f = g modulo some translation in a direction orthogonal to E. 
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Proof. For (/c, n)-Steiner symmetrizations this result is proved in Theorem 10.3 [14]. 
Thus, thanks to the standard inductive argument, we have to prove the theorem for the 
SC 1-symmetrization only. Then — oo < t < oo and we may assume that E = {y = 0}. 
Here x = {x' , y) G M"-^ x M. 

Assume that for some fixed t G M, $7* is not a translation of $7 in the direction of y-axis. 
It follows from Corollary 6.3 that wc can find a polarizer H = {y > to} with to < t 
such that either = 17*, {JhY = /*, 17 / Uh, and anifi) / ftn or (nnY = 17*, 

{/hY = /*! ^ = ^H, f / Ih, and (Tnif) / fn- Then, if w is the solution to the 
problem B2(17if, c, 7, /ij), we conclude by Theorem 9.3 in [14] that 

(11.27) / j{u)dx< / j{w)dx. 

Further, since (17//)* = 17* and (/i/)* = /*, we also have w v. By Corollary 11.3 
this means that 



' j{w)dx< j j{v)dx, 



which together with (11.27) contradicts (11.26). 

Now we assume that for some t G M, 1) = 17* modulo translation in the direction of the 
y-axis. Without loss of generality we may assume that 17 = 12*. Assume in addition 
that / 7^ /*. By Corollary 6.3, there is a polarizer H = {y > ti} with ti < t such that 
Q,H = ^ and f ^ /h, and we can argue as before to derive a contradiction to (11.26). 

Thus, we have f = and it remains to show that /* = g. 

Assume that /* / g. We set / = 7(n) + / and g = 7(1;) + g. Since u = u*, v = and 
u -<* V, it follows from Theorem 11.1 that 

and also 

(11.28) r (fix',s)-~gix',s)) ds<0 

J2t-y ^ ^ 

for all x' G M"~^ and every y > t, where the inequality (11.28) must be strict on a 

subset of 17 of positive measure. Now let h be an arbitrary function in L^(17) satisfying 
h = h*' ^ 0. Then, if w is the solution to the problem Bi(17,c, /i), we conclude that 
w = w*. Moreover, the strong maximum principle yields 

d 



(11.29) 



> a.e. in 17. 



Now after partial integration and by using (11.28) and (11.29) we obtain 

{u — v)h dx = w{f — g) dx 
Jn Jq 



2X 



-K-wix',y) 
dy 



r {fix', t) - g{x', r)) dr) dx'dy < 0. 

J2t-v / 



I2t-y 

Since j' is nondecreasing and u = u^, we have j'{u) = {j'{u)y (see equation (3.6) in 
[14]), and in view of (11.26) it follows that j'{u) / 0. Therefore we may take h = j'{u) 
in the equation above. Because of the convexity of j we get then 

{j{u) - jiv)) dx < j'{u){u -v)dx < 0, 
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a contradiction. The theorem is proved. □ 

Next we prove that solutions of the above considered problems are continuous from 
the left with respect to the parameter of symmetrization t. 

Theorem 11.4. Let (•)*, <t <1, denote the continuous {k,n)-Steiner symmetriza- 
tion into some {k,n)-Steiner symmetrization. Let Q,f,c,j be defined as in Theo- 
rem 11.2 and let t„i, m = 1,2, . . . be an increasing sequence in the standard interval 
I = [0, 1] such that tm ^ to G I. Further, let v and Vm be the positive minimal 
solutions to the problems B2(ri*°, c, 7, /*°) and M2{^^"' ,c,^, f^""), respectively. Then 

(11.30) Vm — >v in W^''^{W). 

Proof, (a) First we prove the theorem for the SC 1-symmetrization. Then —00 < t < 
00. Let tm be an increasing sequence such that tm ^ to e M. Since (^f^^^yo = /*o 
for m = 1,2,..., we may apply Theorem 10.4 to conclude that the functions Vjj-i, 
m = 1,2, . . . are equibounded in VF^'^(M"). Therefore we can find a subsequence Vm' 
and a function w G W^^'^(M"), such that 

(11.31) Vm' w weakly in W^'^iW") 
and 

(11.32) Vm' — >w in L^(M") and a.e. 

By Lemma 5.2, we have C dO,^"" -\-rmBi, where = to—tm- Then rm+i < "Tm for 
m = 1, 2, . . . and — )■ 0. Since f ^ = in R** \ fi*™, we conclude that w G Wo'^(n*o). 
Thus we can argue as in the proof of Theorem 11.2 to derive that w = v. 

Next since (/*'")*"'+i = obtain from Theorem 11.2 that 

(11.33) vi -<*2 V2 ... v. 

Together with (11.32) and the equation (11.25) of Corollary 11.3 this implies that the 
sequence of norms || I'm 111 decreases and 

II l|2 II ||2 

By the uniform convexity of L^(M") this means that 

(11.34) Vm-^v in L^{W'). 

Now (11.34), combined with the fact that the functions Vm, rn = 1,2,... are equi- 
bounded in t^^'^(M"), implies that 

Vm^v weakly in W^'^{R"). 

Finally we have, 

llVUmlli + cllt-mlli = J (jiVm) + f^Vm dx ^ J (^-f{v)-\-f^vdx=\\Vv\\l-\-c\\v\\ 

In view of the uniform convexity of W^'^{W^) this yields (11.30). 

(b) Now we prove the theorem for the continuous (A;, n)-Stcincr symmetrization and 
to = 1- In this case, 17*'^ = Q* and = /*; i.e. fi*" and /*" are just the corresponding 
(k, n)-Steiner symmetrizations of O and /, respectively. Similarly, we have (O*'")* = il* 
and (Z*"*)* = /* for m = 1, 2, . . . Applying Theorem 10.1 [14], we conclude as before 
that the functions Vm, m = 1,2, . . . are equibounded in W^''^{W^). Therefore we can 
find a subsequence Vm' and a function w G VF^'^(M"), such that (11.31) and (11.32) 
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remain valid. By equation (9.5) of Theorem 9.2, we have dd^° C 50*'" + e^Bi with 
some £m > such that — ^ as m — )■ oo. Since f ^ = in M" \ fi*"", we conclude 
that w G P^Q'^(r2*o). As in part (a), this implies that w = v. 

According to Definition 9.1, $7*™+! and are obtained from 0*" and Z*'", re- 

spectively, after a finite number of (k — l,n)-Steiner symmetrizations followed by an 
appropriate SC {k — 1, 7T,)-Steiner symmetrization. Therefore, it follows from Theo- 
rem 10.1 in [14] and Theorem 11.2 of this section that the relations (11.33) remain 
valid in this case as well. As in part (a) of this proof, the latter yields (11.30). 

(c) The theorem is proved for the SC one dimensional symmetrization and for the case 
to = 1, which corresponds to the (/c, n)-Steiner symmetrization. Now, the general case 
follows from these two cases via the standard inductive argument. □ 

Since the proofs of Theorems 11.2 and 11.4 depend only on the maximum principle, 

we can derive similar results for parabolic problems. The proofs of Theorems 11.5 
and 11.6 below are based on the approximation scheme involving solutions of some 
elliptic problems. This idea was used in [5] for the Schwarz symmetrization and then 
in [14] for the {k, n)-Steiner symmetrization. As we will see this method works also for 
continuous symmetrizations. First, following [14], we will define solutions of parabolic 
problems. 

Definition 11.4. Let Q C W be a bounded open set, c > 0, T > 0, f e L\{nx{0, T)), 
if G L\{VI), and let 7 : — )• be a globally Lipschitzian function. We say that u 
solves the problem 1(0, T, c, 7, /, <^) if u is a solution of the following initial boundary 
value problem: 

ueL\0,T;W^'\n))nC{[0,T]-L\n)), ^GL2([0,r];L2(O)), 

(11.35) ut-Au + cu = -f{u) + f m O X (0, T), 
u{x, 0) = ip{x) in O. 

Under the assumptions of Definition 11.4 the problem 1(0, T, c, 7, /, </?) has a unique 
nonnegative solution that can be approximated by the so-called method of discretiza- 
tion in time, see [19] or [14, Section 10]. To define this approximation, we choose 
N G N. Then we divide the interval (0,T) into N subintervals [ti-i,ti\, where 
ti = iT/N, and we set 

(11.36) /.(x) = ^^^' f{x,s)ds, i = l,...,N. 

We put uq = if. Then let Ui be the solution to the problem 

(11.37) Bi(0,c+ (iV/T),7(ui_i) + fi + {N/T)ui.i) 
defined inductively for alH = 1, . . . , AT. 

Let u^{x,t) denote the function of x G O and t G [0, T] defined for ti-i < t < ti, 
i = l,...,Nhy 

(11.38) u^{x,t) = Ui^iix) + {t-ti-i){N/T){ui{x) - Ui-i{x)). 

The latter equation gives the desired approximation to the solution of the prob- 
lem (11.35). Namely we have (sec [19, Theorem 2. 2. 4, p. 42 ff.]): 

u^{-,t) u{; t) weakly in W^q '^(O) for all t G (0, T), 
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weakly in L2([0,T];L2(J7)), 

and 

u^^u mC{0,T;L\n)). 

Theorem 11.5. Let (•)*, < s < 1, denote a continuous {k,n)-Steiner symmetriza- 
tion. Let c, T, f, 7, ip and u be as in Definition 11. Jf. and let g G L^(fi* X (0,T)), 
G ^+(5^^) with /(•, t) g{-, t) and g{-,t) = {g{-, t))' for all t G (0, T), and (p ip, 
tp = tp^ . Let V be the solution of the problem 1(0*, T, c, 7, (/, Then 

(11.39) u{-,t) v{-,t) for all t e {0,T) 
and 

(11.40) v{-,t) = {v{-,t)y for allt e {0,T). 

Proof. Fix < s < 1. For any N & N and 1 < i < A'', let Ui be the solution to the 
problem (11.37) and let Vi be the solution to the problem (11.37) with O, /, and 
replaced by Q^, g, and ip, respectively. By the assumptions wc have uq vq. N is 
large enough then 7(t) + Nt/T is increasing and convex in r. Since / 5, by (11.36) 
we also have /j gi for i = 1, . . . ,N. 

Applying Theorem 11.2, we conclude that Ui Vi for i = 1, . . . ,N. By (11.38) it 
follows that u'^ . Then passing to the limit as AT ^ 00 we obtain (11.39). 

Since Vi{-,t) = {v,{-.,t)Y for all t G (0,T) and alH = 1, . . . , iV we have v^{-,t) = 
{v^{-,t)Y. Passing to the limit as iV — 00, this gives (11.40). □ 

The solutions of the above parabolic problems are continuous from the left with respect 
to the parameter of symmetrization s. 

Theorem 11.6. Let Q., c, T, f,j, ip, and u be as in Theorem 11.5 and let Sm, 
m = 1,2, ... be an increasing sequence in I = [0,1] such that Sm sq E I. Fur- 
ther, let V and Vm be the positive solutions of the problems ,T,c,j,g^° ,tp) and 
I{^}''"',T,c,^,g'^"',ip)), respectively. Then 

(11.41) Vm^v in W^'^iR"") X {0,T). 

Proof. Using discretization in time as in the proof of Theorem 11.5, we approximate 

V and Vm, m = 1,2,... with solutions v^*-* and Vm to the problem (11.37) for an 

appropriate initial data. By Theorem 11.4, Vm — >■ v^^^ in W^^'^(M"). Taking the limit 
as AT — >■ 00 and using Theorem 11.5, we obtain (11.41). □ 

Remark 11.1. The major results in Sections 10 and 11 remain valid if we replace 

the operator (— A + c) by any uniformly elliptic operator which is invariant under 
considered transformations. In the case of the continuous k, ra)-Steiner symmetrization 
this is true for instance for operators of the type 

n—k Q n—k q n q2 

i=l j=l ■' i=n— fc+1 ' 

where the coefficients aij,bi and c are bounded and independent oiy = {xn-k+i-, ■ ■ ■ , Xn), 
c is nonnegative and 

n—k n—k 
i,j=l 1=1 
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12. Appendix 

In this appendix we prove Theorem 3.1. For compact sets O and even j = 2s, equation 
(3.9) is a part of Theorem 4.32 in [25]. If j = 2s — 1 is odd, we apply an even number 
of symmetrizations to the set and again obtain (3.9). 

To prove (3.11) for G J-n, we consider a non-empty shoe J7(x') that is compact in 
Mf'. Applying (3.9) to symmetrizations in slices we obtain 

(12.1) lim d{nj{x'), n*{x')) = 0. 

Since 0.*{x') is a fc-dimensional ball and jC^{0,j{x')) = £'^(yi*{x')), equation (12.1) 
implies (3.11) in the case of compact sets. 

In the rest of this section, we work with open bounded sets Proving (3.10), we may 
assume without loss of generality that all open sets under consideration belong to the 
unit ball 

First we prove three technical lemmas. In all these lemmas, by S we denote the (k, k)- 
Steiner symmetrization in M'^ with respect to the origin x = (0, ... ,0). Then let 
and iS'2 be {k — 1, A;)-Steiner symmetrizations, which approximate S in the sense of 
Theorem 3.1, and let Si and S2 be the symmetry planes (one dimensional) of the 
symmetrizations and ^2, respectively. For notational convenience we will assume 
without loss of generality that Si = {x = (t cos 77r, t sin 77r, 0, . . . , 0) G M'^ : —00 <t < 
00} for some irrational 7 G (0, 1/2) and S2 is the xi-axis. 

For given R, xi, and p such that > 0, —R < xi < R, and < p < R, let 

Z(xi,i?,p) = 4'M(u^Sf (O), 

where the union is taken over all balls 5^ ^^(C) centered at C = (Ci, • • • , Cfc) ^ I^'' such 
that Id = R and Ci = -i'l- 

Lemma 12.1. Let < r < Ri < 1. Then there exists r = T{r,Ri), < r < 1, such 
that for every compact set K satisfying the conditions K C -B^^ for some R G [Ri, 1] 
and C^{K) < C^{Bi^^) there is real xi = xi{K) such that —r<xi<r and 

B^^\K2 D Z{xi,R,tR). 

Proof. Since K2 is Steiner symmetric with respect to S2 and since 
there is a point xq = {x^, X2,0, . . . ,0) G Sr'^^ \ K2 such that — r < x\ < r and x^ = 
- (xy)2 > 0. 

Since the slice ^2(3^1) is a (A; — l)-dimensional ball we have K2{x1) C b'^o~^\ Since at 
the same time K2{xi) is the result of S2 symmetrization of Ki{xi) we have 

(12.2) £'=-\4')(x?) \ K,) =C^-\B^^\x\) \ K2) 

>£'=-i(<(x?)\s3-^))>c(r,i?i), 

where 

c(r, i?i) = rniii C^-\Bf^{x\) \ B%-'^) > 0. 
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For X G B^^^ \ Si, let ri(a;) denote the ray through x that is orthogonal to Si and has 
its origin at some point 5 G Si fl B^^\ Let I{x) be a closed segment of ri(x) joining 
X and dB^^\ We note that I{x) D Ki = $ ii x ^ Ki. Let 

J(x?) = Uj{x), 

where the union is taken over all X G B^^\x^) \ Kiix^). Let A = A{K,x^) be the 
maximal segment of the xi-axis such that x^ e A and if xi G ^ then 

(12.3) ji:''-\R''{xi) n J(x?)) > (l/2)c(r, i?i). 

Using (12.2), one can show that there is a constant t = t{r, Ri) > such that 

(12.4) jC\AiK,x^^))>t{r,Ri) 

for every compact set K satisfying the assumptions of the lemma. 

Finally, it is not difficult to see that (12.3) and (12.4) imply the lemma. □ 

The following lemma allows us to control how fast the approximation process is for 
compact sets. 

Lemma 12.2. Let < r < i?i < 1. There exist a positive integer N = N{r,Ri) and 
real (3 = /?(r, Ri), < /3 < 1, such that for all j > N and all R such that Ri < R < 1, 

we have 

K r R^^^ 



for every compact set K C such that C^{K) < C^{B^^^). 
Proof. We may assume without loss of generality that 



(12.5) K (1B^^\z{xi,R,tR) 

with T = r(r, -Ri) > defined in Lemma 12.1 and some xi = xi{K) such that — r < 
xi < r. Indeed, if K does not satisfy (12.5), then we replace K with its second 
symmetrization K2, which by Lemma 12.1 satisfies the required condition. 

Let Cr be the circle in the plane = {x G M*^ : x = (xi, X2, 0, . . . , 0)}. Using complex 
notation, we will write Re^^ for x = (ii cos 6*, i? sin 0,0, ... ,0) G Cr. 

It follows from (12.5) that there is a sufficiently small constant oq = aQ{r, Ri) > such 
that for every compact set K satisfying the conditions of the lemma there is a point 
Re^^° with ^0 = Go{K) such that the intersection Cr fl Z{xi,R, tR) contains an arc a 
centered at i?e*^" with the angular measure > oq. 

In addition, it is not difficult to sec that there exists a constant v = v{r, Ri), < < 1, 
such that the set CrH Z{xi, R, utR) contains a subarc a' <Z a again centered at i?e*^°, 
which has the angular measure > (2/3)ao. 

Let Refi and Ref2 denote the reflections in the plane £ with respect to the lines 

li = {z = te^'^'^ : —00 < t < 00} and I2 = {z = t : —00 < t < 00}, respectively. Let 
7^* = (Refi o Rcf2)^ s = 2,3, . . .. Then 7^*(i^e*''«) = i?e*(2*T"^+^o), s = 2,3, . . .. 

Since 7 is irrational, the set {Re'^'^^'^^}'^2 is dense in Cr; see, for example, [25, 
Lemma 3.25]. Therefore there are indices si, . . . , sjvi such that for any the points 
2^gi(25,-77r+eo)^ j = 1, . . . , AT^, divide Cr into A^i arcs, each of which has the angular 
measure < ao/4. Let N = maxjsi, . . . , sjy-^}. Let 

Q;s = 7^^(a), a^ = 7^^(a'), s = 2, 3, . . . , A^, 
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where a and a' are the arcs defined above. By our choice of the points Re^^'^^^'^^+^o) ^ 
we have 



Let 



W{xi) = \ Z{xi,R,tR), W'{xi) = bP \ Z{xi,R,iyTR), 



where xi = xi(K) and r and v are positive constants defined in the beginning of this 
proof. For s = 1, 2, . . . and xi = xi{K), let 

Wsixi) = {S2 o SiYiWixi)), Wi(xi) = (^2 o SiYiW'ixi)). 
It follows from (12.5) and our construction that 

K2N C Wn{xi) C Wj^ixi) 

and 

(12.6) W'j^ixi) n = 0. 

Let 



/3(xi) = inf{^ > : Wn{xi) C ^(fc)^^}. 

By (12.6), < /3(xi) < 1 for all xi such that — r < xi < r. Let j3 = sup|^^|<^ /3(xi). 
Our construction above works for every compact set K satisfying the assumptions of 

the lemma. Choosing K = Br^\ we obtain that >r/R>r > Q. 

Thus, to complete the proof, wc have to show that /3 < 1. If not, we can find a sequence 
x^*^ G [— r, r], s = 1,2,..., such that x^*'' — )• x^'^'* and ^(x^*'*) — >■ 1 as s — )• oo. For all 
sufficiently large s we have 

Z{xf\R,vTR) C Z{x^'\r,tR). 
Therefore for all such s we have 

W'^ixf^) D Wn{x['^). 



By (12.6), W'^{x^i ^) C Bj^,^j^ for some < P' < 1. Hence for all sufficiently large s we 

have 

W^{x^^^) C bP^ 

contradicting the assumption that /3(x'^*)) 1 as s ^ oo. The proof of the lemma is 
complete. □ 

Lemma 12.3. Let S, Si, and S2 be the symmetrizations as in Lemmas 12.1 and 12.2. 
Let < r < R and < p < R. Then there is a constant ci = ci(r, p,R) > such that 

(12.7) £'^(17i\4^')) > ci 

for every open set C M'^ such that Q2 contains some point xq = {xQ,yo) with Xg G M, 
yo £ M*^"-^ such that |xo| > R and \yo\ > p. 

Proof. Let O satisfies the assumptions of the lemma and let 

02 = C2{r,p,R) = minC''-\{{x'o} x ij('=-i)) \ i?W), 

Xq 

where the minimum is taken over all Xg such that (xg)^ + p"^ > R^- It is clear that 
C2 > 0. Since VL2{xq) is a (A; — l)-dimensional ball with the radius > p, we have 

C^-\n2{x'^)\B^'''^)>C2. 
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For a; ^ El, let I{x) denote the segment orthogonal to Si that joins x and Si. For 
X eR''\ {B^''^ U Si), we set I{x) = i{x) \ B^''\ We note that I{x) C Oi, whenever 
X e Ui. For Xq G M defined above, let 

where the union is taken over all X = {x'^,y) eR''\ B^!'^ such that y G rii(x'o). Let 
Ai{i},XQ) be the maximal closed segment of the xi-axis such that x'q G Ai(yi,XQ) and 

(12.8) C''-\iMx'o))ix^))>C2/2, 

where (Ji(xg))(xi) denotes the {k — l)-slice of Ji{xq) at xi. It is not difficult to see 
that there is a constant ti = ti{r, p, R) > such that 

(12.9) C\A^{Q,x'o))>h 

for every domain Q satisfying the assumptions of the lemma. 

Since Ji{x'q) C fti, equations (12.8) and (12.9) imply (12.7). □ 

Proof of Theorem 3.1 for open sets. The proof is by contradiction. Suppose that 
equation (3.10) does not hold for some open set Q C B^^\ Then we have to consider 
the following two cases. 

(i) There are real number eo > and a sequence Xg — >■ xq such that Xg G dVLms for 
some subsequence of indices nig and 

(12.10) d{xs,d^*)>eQ for s = 0,1,... 

(ii) There are real number £o > and a sequence Xg xq such that Xg G dO.* for all 
8 and there is a subsequence of indices rUg such that 

(12.11) d{xs,dQms)>eo for s = 1,2,... 

In each of these cases we may assume without loss of generality that all indices vrig are 
odd and rrig > 3. Then fi^^ = S'i(r2^^_i). 

To prove (i), we consider two subcases. 

(1) We first assume that si"^(xo) C Jl*. Let = (x^, y^), s = 0, 1, . . . Then there are 
constants (5o > and R> |?/o| such that 

(12.12) n*{x')^Bf ioi sl\x' ^Bf-^\x'Q). 

Next we estimate how much the slice ilms-i{x'g) differs from the ball b'^^\ Let i^i(x', y) 
and L2{x',y) be the symmetrizing planes through (x',y) G M"~'^ x M/^ of Si and S2, 
respectively. Then Lj(x',y) can be represented in the form Lj(x',y) = {{x',t) : t G 
Lj{y)}, where Lj{y) is a (A; — l)-dimensional plane in M'^, which does not depend on 
x'. 

Since yg — )■ yo and R > \yo\, we have that \ys\ < Ri = {R+ |yo|)/2 for all sufficiently 
large s. Since f7„j^ is the {k — l)-dimensional Steiner symmctrization of ^j^s-i with 
respect to ^i, the set r2m^(x^)nLi(ys) is a (fc — l)-dimensional ball in the corresponding 
(A: — l)-dimensional plane. Since {x'g,ys) G 00,^^, the latter implies that 

QmAO n Liiyg) C n Li{yg) C i^J^ n Li(y,). 
This implies that there is 61 = Si{R, \yo\) > such that 

jC'^-\iB'^^^nLi{yg)) \ inmMnLiiyg))) > 61 
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for all sufficiently large s. Since D,ms = Si{Qms-i)j the latter inequality implies that 

(12.13) JO''-\{B'j^^ n \ i^ma-iix's) n > 5i 
for all sufficiently large s. 

Now since Qms-ii^'s) omits a set of positive (/c— l)-dimensional measure in B^^^r\Li{ys) 
and at the same time ^lms-iix's)f^L2{ys) is a (/c — l)-dimensional ball in ^2(2/5), we can 
argue as in the proof of Lemma 12.3 (cf. how (12.7) follows from (12.8) and (12.9)) to 
deduce that there is 62 = 52{R, \yo\) > such that 

(12.14) £\B'^^\nma-i{x's))>S2 
for all sufficiently large s. 

Next wc show that (12.14) leads to a contradiction. The slice ^{xq) is an open set in 
B^''\ Therefore for arbitrary small e > there is a compact set F C ^{x'q) such that 

(12.15) £''{n{x'o)\F) <e. 

Then for sufficiently small ei > 0, we have B^^ ''\xo) x F G By the monotonicity 
property of symmetrizations, we have 

where (({.Xq} x F),„^._i)(.Tq) denotes the slice of ({xg} x F)^^-! at x'q. Applying 
equation (3.11) of Theorem 3.1 to the compact sets {x'} x F and {x'} x S(F), we 
obtain the following limit relation for measure in slices 

(12.16) CHmx'} X F)j){x')) A {{{{x'} x S(F)))(x'))) ^0 as j ^ 00. 
Now, (12.12), (12.15), and (12.16) imply that 

/:'=(i?f)\((K}xF)„,_i)(x'J)<25 

for all sufficiently large s. Since {{{x'g} x F)jns-i){Xs) C Q,ms-i{x's)j the latter inequal- 
ity contradicts (12.14). This completes the proof of the theorem in the case under 
consideration. 

(2) In the second case, we assume that ^^"■'(xo) fl fi* = 0. We recall that Xg = 
{x's, Vs) xq = {x'q, yo) as 00. 

If yQ = (0, ... ,0), then Vt*{x') = for ah x' G M"-"^ sufficiently close to .Tq. Hence, 
^l{x') = and therefore ^j{x') = for all such x' and all j = 1, 2, ... It is easily seen 
that the latter contradicts our assumptions that Xg G f^^^m^ and Xs — xq as s — )■ 00. 

Assume now that |yo| = -Rq > 0- Since B^eo{xQ) n ri* = there is (Jq > such that for 
every x' G ^''(■^0) the slice il*(x') is either an empty set or an open /c-dimensional 
ball B^j^^^,^ with the radius R{x') such that 

(12.17) < R{x') <p<Rq 

with some p independent of x' G B^^ ^\x'q). In particular, (12.17) shows that 

(12.18) < d'inix')) = c''{n*{x')) < d'iBf'^) 

for all x' G ^^''"''^(x'o). 
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Let £0 > be fixed and sufficiently small. For every x! G i?^^ '^^(^o)) we choose a 
fc-dimensional compact set K(x') such that K(x') C J7(:r') and 

(12.19) &{^{x')\K{x'))<eQ. 
By (12.18), we have 

(12.20) &{K{:^))<&[Bf) 



for all x' G Bf~^\x'^). 

Let £2 > be sufficiently small and let £i > be such that p + £i < i?o and 
(12.21) L\B%^\Bf)<e^ 
for all p < Ro- 



For j = 1,2,... and x' G B^^~''\xq), let K^{x') denote the slice at x/ of the j-th 
successive symmetrization of the set {x'} x K{x') defined by formulas (3.7) and (3.8). 
Alternatively, K^{x') can be obtained by applying appropriate {k — l)-dimensional 
symmetrizations to the set K{x') in R*^. 



Since K{x') C for all x' and since K{x') satisfies (12.20) for aU x' G fij" ''\x'q), 
we can apply Lemma 12.2 with r = p and R\ = p + £\\,o the compact sets K{x'). This 
implies that there exists a positive integer N = N{p,£i) such that 

(12.22) K^{x') C bI%^ 



for all x' G ''\x'f^) and all j >N. 

Now, combining (12.19), (12.21), and (12.22), we obtain that 
(12.23) C^{nj{x')\B^''^)<eo + e2 



for all x' G ''^(x[,) and all j > N. 

Now we return to the sequence Xg = {x'g,ys) xq = (xQ,yo)- Since Xs G dQ^sJ for 
every s we can find a point Xg = {x'..,yg) G f^ms such that — >■ as s — >■ oo. Then, 
of course, ^{x'g) ^ and therefore Q*(x'g) ^ 0. 

Suppose first that xq S2. Then d = d{xo, S2) > 0. Now we can apply Lemma 12.3 
with r = p, R = (i?o + p)/2, and p = d and with the domains J7i and ^2 in that lemma 
replaced by the domains Qms-ii^'s) and J7„i^(.t^,), respectively. By Lemma 12.3 there 
exists a constant ci = ci(p, d, Rq) > such that 

/:^(o„_i(x^)\sW)>ci 

for all sufficiently large s, which obviously contradicts (12.23) if Eq and £2 are chosen 
sufficiently small. This proves the theorem in the case under consideration if xq ^ S2. 

Suppose now that xq G S2. For every s = 1,2,..., the symmetrizing plane L2{ys) 
contains some point Xg = {x'g,ys) £ d^ma-i- Selecting a subsequence if necessary, we 
may assume that Xg x^^^ = (xq, y^-*^)). 

Since B^\xq) fl $7* = and xq G S2, it follows from the definition of (A;, n)-Steiner 
symmetrization that 

si^)(xW)nJ)* = 0. 
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Now, if x^^^ ^ Si, to complete the proof, we can apply our argument above replacing 
the plane S2 and the sequence of points Xs G dQ^s with the plane Si and the sequence 
Xs G dilms-ii respectively. 

In the case x^^^ G Si, we continue our construction to find points x^^^ € S2, x^^^ € Si, 
. . . The sequence of points x^^^ will be finite if x^"^"^^ S2 or x^'^"^-'^) Si for some 
m > 1. Otherwise, the sequence x^^^ will contain infinite number of terms. If it is 
finite, say j = 1, . . . ,N, then we apply our previous argument to the point x^^^ and 
to the plane Sj, where i = 1 if is odd and i = 2 if is even. 

Assume now that the constructed sequence of points x^^^ = (xq, y^^^), j = 1,2, . . ., is 

infinite. By our construction we have ly^-^^^''! > ly^-^'-'l sec(77r) for all j = 1,2, 

Therefore, 

\x^^^ \ — )■ 00 as J — )■ 00. 
Since for every j there is an index m{j) and a point z^^^ G ^m{j) such that \z^^ —x^^^\ < 
1 the latter limit relation contradicts our assumption that Q C B^^\ 

This completes the proof of the theorem in the case (i). 

In the case (ii) the proof is simpler. As in the case (i), we consider two subcases. 

(1) First, we suppose that Bi^\xo) D Jl^^ = for some sufficiently small £0 > and 
some infinite subsequence of indices m^, s = 1, 2, . . . 

Fix 5o > sufficiently small. Then let K be a compact subset of such that 

(12.24) £''{n\K)<5o. 

Let K* = S{K) and let Kj, j = 1,2, . . ., be successive symmetrizations of K defined 
by (3.7) and (3.8). Since Km, C n rris ; we have 

(12.25) B^'^Xxo) n Km, = $. 
Since xq G 90* there is (5i > such that 

c^iB^\xo)nn*)>6i. 

This together with (12.24) implies that 

(12.26) C^{B^Hxo)nK*)>52 
for some (^2 > if (5o > in (12.24) is sufficiently small. 
Applying equation (3.11) to the compact set K, we obtain 

(12.27) jO-^'iKms AK*)^0 as s ^ 00. 

One can easily see that equations (12.26) and (12.27) contradict (12.25). This proves 
the theorem in the case under consideration. 

(2) Suppose now that £0 > and xq = {xQ,yo) are such that 

(12.28) B(^\xo)cnm, 

for some infinite subsequence m^, s = 1,2,... This implies that, Clmsi^'o) 7^ ^ ^i^d 
therefore ^(x'q) 7^ and Q*{xq) 7^ 0. Since r2*(xQ) is a /c-dimensional ball there is 
r > such that 0*(xq) = Bi^\ Since xq ^ ^l*, we have r < |j/o|. This implies that 

(12.29) C'^i{B(-\xo)\n*)ix',))>6, 
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for some di > 0. Here {B^oH^o) \ ^*){x'o) denotes the slice of Bi'^\xo) \ ^* at x'q. For 
every arbitrary small ^2 > there exists a A;-dimensional compact set K C ^{x'q) such 
that 

(12.30) C'inix'o) \ K) = c\n*ix'o) \ K*) < 62, 

where K* denotes the (k, fc)-Steiner symmetrization of K. 
Applying equation (3.11) of Theorem 3.1 to K, we obtain 

(12.31) jC'^iKs AK*)^0 as s ^ 00, 

where Kg = {{{xq}xK)s){xq) denotes the slice at Xq of the symmetrized set {{xq}xK)s 
defined by formulas (3.7) and (3.8). 

By the monotonicity property of symmetrization, we have C ^^^(xo). This and 

(12.30) imply that 

(12.32) C''{nmA4)\Krns) <S2- 

Finally, (12.28), (12.29), and (12.32) imply that there is a constant ^3 > such that 

(12.33) C''{KmA^*{4))>^3 

for all sufficiently large s. Now one can easily see that (12.33) contradicts (12.30) and 

(12.31) if (52 > in (12.30) is sufficiently small. 

The proof of equation (3.10) of Theorem 3.1 is finished. 

To prove (3.11) for Q G Gn,b^ we fix x' G W^~'^ such that the slice O(x') is not empty. 
Considering the restrictions of the symmetrizations S, Si, and ^2 to the slice R"'(x'), 
we obtain 

(12.34) lim d{dnj{x'),dn*{x')) = 0. 

Since fl*[x') is an open k dimensional ball and C^{^l*{x')) = C^{Qj{x')), equation 
(12.34) implies (3.11) in the case of bounded open sets. 

The proof of Theorem 3.1 is now complete. □ 

Remark 12.1. One can easily show that (3.11) remains valid even for unbounded 
sets $7 if the measure of the corresponding slice is finite, i.e. if >C*^(ri(x')) < oo. 

In contrast, simple examples of unbounded domains with a finite measure, £"(0) < 
00, show that (3.10) is not true for unbounded open sets in general. 
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